The first interment was that of Lucretia Hammond of Kernville on April 30, 1887.

The cemetery initially was known as Grand View Cemetery. The trustees selected the name to reflect the "grand view" of the city.

One-hundred acres on Yoder Hill west of the city was purchased from the Cambria Iron Co. The company had used the site as pasture and farmland for horses and mules used in its mills and mines.

In the late 1880s, a winding carriage road over one-mile long was constructed up Yoder Hill with two imposing stone arch memorial entrances. Near the upper arch an administration building and maintenance garage were erected in 1897. Soon afterward it was found that an entrance to the cemetery from the Westmont side would be necessary. In 1904 this entrance was made at Bucknell Ave., along with the building of a stone fence extending along Millcreek Road, a distance of 3,520 feet, for protection of cemetery grounds. In 1991 this wall was extended an additional 2,530 feet along Millcreek Road to Menoher Boulevard. Its design, of cut stone columns connected by black aluminum rods along the winding shape of the road creates an artistic looking boundary for this section of the cemetery.

In the 1940s the old roadway was abandoned with the construction of Menoher Boulevard or the "Easy Grade Highway" as it is better known. A new, modern administration building and maintenance garage replaced the old structures in 1989.

Most of the cemetery's historical significance comes from the Johnstown Flood of 1889. Many of the Flood's 2,209 victims are buried there. A section called the "Unknown Plot" Contains the bodies of 777 flood victims who could not be identified. The state Flood Commission purchased the plot for burying the unidentified victims. The commission bought markers for each grave and a monument that cost $6,500. This monument, with its immediate surroundings, forms probably the most striking feature of the cemetery. It was dedicated May 31, 1892, in the presence of about 10,000 people, among them the Governor of the State and many prominent men from all over the Commonwealth.

During the Johnstown Flood Centennial of 1989, 700,000 people from all over the state, nation and world visited the Johnstown Flood Memorial near St. Michael, the Johnstown Flood Museum in downtown Johnstown featuring the Academy-Award-winning film about the flood by Charles Guggenheim, and the Unknown Plot at Grandview Cemetery.

Thousands of people ask each year for directions to the cemetery and the locations of gravesites of relatives and friends.

The cemetery contains the grave of Lt. Col. Boyd "Buzz" Wagner, the first American air ace of World War II. Also buried in Grandview is John G. McCrory, founder of the McCrory 5-and-10 store chain.

The number of persons at Grandview as of March 31, 1992 was 57,006. In 1977, three masoleums were added to the cemetery. To this day, the names of all persons interred in Grandview are hand written in the Chronological Book of Interment.

Containing 47 burial sections and more than 235 acres, the cemetery is one of the largest in Pennsylvania.