In the 1860s, Father Peter McCoy, Pastor of St. Lawrence Church in Locust Point (now Our Lady of Good Counsel see below), realized that there was a need for an additional church in the South Baltimore area referred to as "the Hill." At that time, the surrounding area included open country, an undeveloped suburb, a few farming spaces and countless clay banks. The view westward from Fort McHenry was practically unobstructed and there were no car lines connecting this part of Baltimore with the rest of the city.

In the fall of 1868, Father McCoy purchased the present site of Saint Mary, Star of the Sea from a Mrs. Arnold. The first building erected by Father McCoy was a rectory of red brick. The basement of this rectory, which served the congregation for three years as a church, was about 50 feet long and 20 feet wide. Later it was used as a school room and assembly hall. The first Mass was celebrated in this basement church on December 16, 1868.

The cornerstone of the church was laid on May 9, 1869. Two years later, on March 26, 1871, the church was dedicated by the Most Reverend John Martin Spalding. St. Mary, Star of the Sea was established to meet the spiritual and material needs of the growing Irish immigrant population, who found work in the very active shipbuilding industry and with the Baltimore & Ohio Railroad.

As the parish grew, a tall spire with a light at the top was added to the Church building. Originally powered by candles and mirrors and later by electricity, this light has served as a beacon to ship traffic in the Inner Harbor and as an official landmark on mariners maps of the Port of Baltimore. In the mid 1890s, beautiful stained glass windows made in Munich were added to the church, and in the 1920s a magnificent Cassavant organ was purchased and installed. In 1903, weather wreaked havoc at St. Mary's when a disastrous tornado swept through Baltimore and toppled half the steeple. Despite this damage the sanctuary remained opened for Sunday services. Nearly a century later, in 1999, St Mary's again experienced roof problems when shifts in the wood support beams caused the roof to slide off its foundation and leave a four foot hole in the roof.

St. Mary, Star of the Sea school was begun in 1877. In 1914, a new school building was built at Battery Avenue and Gittings Street, and was staffed by the Sisters of St. Joseph. In 1972, the school merged with the previous Holy Cross and Our Lady of Good Counsel schools to become the South Baltimore Area Community School. In 1982, this consolidated school became the Catholic Community School of South Baltimore.

On March 26, 2001, the celebration of the Feast of the Annunciation, St. Mary, Star of the Sea parish celebrated it 130th anniversary. More recent pastors at St. Mary's included Father Jack Lippold in the 1980s, Father Glenn Byrne in the early 1990s and current Auxiliary Bishop of Baltimore, Most Reverend Mitchell T. Rozanski in the late 1990s. Fr. Tom Malia became pastor in 1999 and he was succeeded by Father Ray Martin in 2002, pastor of Our Lady of Good Counsel. It was at that time the three parishes of Holy Cross, St. Mary, Star of the Sea and Our Lady of Good Counsel came under direction of one leader, forming the Catholic Community of South Baltimore, though each still a separate canonical parish. Fr. Patrick Carrion succeeded Fr. Martin in January 2008, Fr. Joshua Laws became the parish administrator in July 2019, followed by Fr. Kevin Ewing in 2024.