In former times, in the section of the Central City Cemetery in Sumy set aside for Jews, only Jews were buried behind a separate fence. But over time, everything changed, not for the better, and now next to the matzevot with the Star of David you can see tombstones not only with Christian, but also communist symbols.

Today there is no exact information about when the Sumy Jewish cemetery was founded. For the first time, as a separate object, it was indicated on cadastral maps of the city at the end of the 19th century, issued during the times of the Russian Empire. According to the dates preserved on many old tombstones, the burials were made approximately 120-130 years ago. It is assumed that somewhere during this period a cemetery was organized for the Jews of the then small Sumy community. Historical evidence suggests that there were few Jews in the city at that time, as it was outside the officially designated Jewish Pale of Settlement.

Today the length of the perimeter of the Jewish part of the cemetery is approximately 400 meters. Its territory is quite large and, for the most part, well-kept, with mowed grass, pruned bushes and old trees.

More than 700 tombstones of the approximately 1,500 that exist at the Central City Cemetery in Sumy, are Jewish. Many, especially the oldest matzevot, have almost lost the inscriptions and signs by which one could recognize who was buried under them. Some of the old tombstones were destroyed under the influence of atmospheric phenomena and nature. On earlier, especially post-war tombstones, which were made according to Soviet traditions, the inscriptions were well preserved, many of them are duplicated in Russian, but there are many that contain only Russian inscriptions.

Thousands of local Jews were tortured and shot during the city's occupation by German troops during World War II. Moreover, the most massive execution of Jews took place, by tragic coincidence, in a ravine area just not far from this cemetery. In memory of this tragedy, a monument was erected at the Jewish cemetery in Sumy.

According to representatives of the local Jewish community, the appearance of new graves of non-Jews on the Jewish site next to Jewish matzevot (and sometimes in their place) violates Jewish canons regarding the sanctity of this place. However, so far the Jewish community of Sumy has not been able to influence this, and requests for the allocation of a new site also remain unheeded. Therefore, its efforts are aimed at looking after and cleaning the old site and the Jewish graves located on it.