The Bukovje Manor House rests in the natural surrounding of trees, nearby forests, and the River Drava. The manor house is part of the former Bukovje lordship, to which Pukštajn Castle (also Puchenstein Castle, of which the area takes its name) and the Our Lady of Sorrows chapel belonged (the chapel being the entrance point to the Bukovje Manor). The park begins a few hundred metres away from the manor house, and over it rise the ruins of the former Pukštajn Castle.

The Pukštajn, or Bukovje Lordship was established in the 13th century by the Counts of Andechs-Merania. Because of the proximity of the Drava, the Mislinja, and the Meža rivers, the castle had a strategic role due to an important trade route running along the River Drava. Maribor and Klagenfurt (Celovec) could also be easily reached by land. The destruction of Pukštajn Castle, which now remains a ruin, made way to the building of the new Baroque manor house, now known as the Bukovje Manor House. The manor house soon became the seat of the feudal and county lordship, as well as the seat of the regional court. Its last owners, the Counts of Kometter (also Kometer, or Kemeter), came into possession of the manor house in 1817. Janez Krstnik, Baron Kometer, rebuilt the manor house in 1870 in accordance with historical traditions, and it thus became one of the most important monuments of castle architecture in the area. After World War II, the manor house and the adjoining park were nationalised and became the property of the Yugoslav National Army. The park was turned into military training grounds, the manor house was renovated, and extensions were built to meet the needs of the military. By doing that, the rich interior of the house was unfortunately lost. After Slovenia's War of Independence, the manor house was used by the Slovenian military.

In 2008, the Ministry of Defence transmitted the manor house and a part of its park to the Municipality of Dravograd. Now, the manor house seats the Dravit Dravograd Public Institute (the institute for sport, tourism, culture, and youth).