Officially, Jews received the right to settle in Zhytomyr from the beginning of the 18th century. By the end of the century, they accounted for a third of the population, and by the end of the 19th century already 46%. According to local historians, at the end of the 18th century there were three cemeteries in the city: Orthodox, Catholic and Jewish, and they were located in the area of ​​Okhrimovaya Gora. By the twentieth century, they all ended up within the city and disappeared. The House of Officers was built on the site of the Jewish cemetery in the 1930s.

The cemetery, which local residents called the Old Jewish cemetery, arose around the end of the 18th century. Probably, its appearance is connected with the closure of the cemetery on Ohrimovaya Gora. The first burials in the cemetery appeared at the beginning of the 19th century. The cemetery was located in the area of ​​Bolshaya Berdichevskaya Street, not far from the modern Smolyanka microdistrict. The cemetery operated until 1976, but the possibility of additional burials remained, since by that time the share of the Jewish population in the number of townspeople had dropped to 3.7%. The area of ​​the cemetery, according to city registers, is 24 hectares. There are more than four thousand graves on it.

The cemetery was partially damaged by the actions of the Soviet government. In its depths are the graves of victims of political repression of the 1930s-1940s. During the Second World War, the cemetery suffered from the actions of the occupiers, and in the 1970s, part of the cemetery was given over to a garage cooperative. According to local historians, burials dating back to the 19th century suffered the most.

Nevertheless, the crypts of a number of Hasidic tzaddiks have survived in the cemetery. In particular, Rabbi Aharon of Zhytomyr, who was one of the students of the founder of Hasidism, Baal Shem Tov. The graves of rabbis have also been preserved at the cemetery:

• Mordechai from Kotelny (died in 1913).

• Mordechai from Korostyshev (date of death 1916).

• Moshe from Korostyshev (buried in 1920).

• Yakov from Korostyshev (died in 1941).

The cemetery is interesting in that you can find grave monuments in the form of chopped off trees, as well as traditional matzevas with inscriptions and epitaphs in Hebrew.

There are more than 260 graves in the cemetery, where anthroponymical and chronological data have been preserved only partially. The earliest of them date back to the 19th century and contain inscriptions exclusively in Hebrew. In particular, Shimon Gershon (died 1895), Shmuel Aizikovich (died 1894), and Tsvi Shmuel (died 1894).