The Samuel Miller house is an 1820s Federal-style frame house nestled inside the neighborhood of Richland Hills. Unassuming, the exterior belies a rich history of Civil War skirmishes, family intrigue and mass amounts of wealth.
In the 19th-century, it was the main house of a 1,500-acre plantation, home to Samuel Miller, one of the wealthiest men in the antebellum South, and a notorious recluse.
Always, it seems, the property is a master class in contradiction. Despite Miller's wealth, the home was unpretentious and devoid of adornment, even in its heyday. Miller, a miser in his lifetime, was a generous philanthropist after death.
Now, despite its traditionalist exterior, with white wood siding, a sprawl of green lawn and towering trees, its interior is eclectic and modern.
There’s a life-size model skeleton grinning from the corner of Richard Gilman’s study. And Julie Bryant’s art studio is flooded with light, large, colorful canvasses leaning in stacks. In the bright orange foyer, a gilt-framed portrait of Miller, sporting a somewhat dour expression, hangs on the wall.
"I mean, ultimately, he was kind of a freaky dude." That's Richard's take on Miller, who never married but kept a mistress — by some accounts scandalously young — named Mary Davidson. They had five sons, though he made no special provisions for the education of the children or assimilation into society, according to the nomination form for the National Register of Historic Places.
The Samuel Miller house was registered as a Virginia Historic Landmark in 1992.
A tobacconist, merchant and railroad investor, Miller lived from 1792 to 1869. Born near Batesville in Albemarle County, he was fatherless and raised in abject poverty. He moved to Lynchburg with his brother as a young adult, and they both prospered as tobacco and grocery merchants. A substantial amount of his wealth came from railroad stocks and state or municipal bonds. He spent the last 40 years of his life in a "self-imposed isolation" in his Lynchburg home, where he lived until his death.
Julie said they believe he was a hypochondriac who at 37 began to be plagued by a chronic affliction.
According to the nomination form, whether his illness had any medical basis or whether it was psychosomatic, it "dominated his life."
During his reclusion, he spent much of his time in a sun parlor on the second floor. Considered an unusual house addition during the period, the nomination form posits it contributed to the local gossip about his eccentricities.
Julie and Richard bought the home in March 2020. Thought they had no concrete plans to buy, when they first saw it they "just knew."
"Have you seen it? It's fabulous," said Julie. She is a social worker with Centra Hospice, and Richard is an attorney. He also hosts a radio show on WLNI, "Law Dog Radio," billed as "legal information with a twist."
Julie likes that the house feels lived in. With no real stylistic intention, she said they decorate it with what feels right, an amalgamation of their interests, which lends itself to a bright, comfortable home. Many of the rooms are punctuated with ornate light fixtures — from parrot-shaped stained glass, to delicate tear-drops or spiky, contemporary pieces.
In their favorite room, now called the den, the light fixture is a draping, thick rope, which hangs from the bare beams over a low, plush leather couch. That room was once the detached kitchen, which would have been used until the home's extensive additions in the 1930s. It still bears slightly leaning walls, and a utilitarian brick floor.
"You're connected to history, living here," said Richard. He was in the den, and gestured at the exposed brick of the large hearth. "They were cooking food here when they built the house. [Miller] lived in the cottage, he walked through here.”
The cottage he referenced sat about 50 yards west of the main house and was where some believe Miller lived while the home was constructed. It also likely served as the domicile of the property's overseer. Though it likely predated the house, the year it was built is unknown.
One of several outbuildings on the property, Miller had as many as 17 slaves, all believed to have been emancipated before the war, with several remaining as hired servants.
The original house, built between 1826 and 1829, was two and a half stories. It had a T-shaped floor plan, with two adjoining gabled roofs and featuring three third-floor dormers, clapboard siding and three brick chimneys.
A detached kitchen was erected about ten feet to the west. It later was attached to the house by a small frame hyphen, according to the nomination form.
The original floor plan showed a first floor made up of a parlor, dining room and library. The second story contained three bedrooms, and the third-floor attic housed a finished fourth bedroom, and a garret, with dark exposed beams and slats through which you could see the tin roof above.
The 1938 renovations made several significant additions and expansions, including updates to install indoor plumbing, radiators and a central furnace. The renovations enclosed most of the first floor porches, extended two bedrooms, added bathrooms and attached the detached kitchen. A modern kitchen and laundry room were introduced on the southwest side, relegating the original kitchen to its current use as a den.
In the 1950s, a sun room with ceramic tile floor, now Julie's studio, was extended from the southwest corner of the house, and a nuclear fallout shelter was hand-excavated beneath the center hall and dining room. Still attached to the cellar wall is what Julie called a "fresh air return," with a crank-handle that could be spun to pump fresh air into the tight space.
As the Cold War escalated in the 1950s, families across the country built fallout shelters in their basements and backyards. The fallout shelter was connected to the home's long, narrow cellar — one of the other 1930s additions — and access was gained from the first floor via a descending staircase under the original staircase.
Julie said they don't have plans for many renovations themselves, just what "needs to be fixed." There are some improvements they would like to do in the kitchen and master bathroom and, in the coming years, they want to repaint the exterior. They hope to have any major fixes done by the house's 200th anniversary in 2026.
Sitting in relative isolation, what then would have been about five miles beyond the outskirts of Lynchburg, an island in the midst of the sprawling plantation, during the 1864 Battle of Lynchburg a cavalry skirmish took place on the property, and Union troops pillaged Miller's home.
As the story goes, Miller took precautions before their arrival and buried gold coins and important financial certificates around the property in waterproof cylinders. Some would not be recovered until his death several years later.
"We haven't found any," interjected Julie.
"Well, we haven't really been looking," Richard added.
Pointed toward the house by his neighbors, Union troops confronted a bedridden Miller in his upstairs chamber. He refused to disclose his hiding places, and challenged them to shoot him, declaring he "had little time to live anyway."
According to one account, he was not intimidated when a musket round was discharged into the ceiling above his head.
In Richard's account, Miller knocked the barrel of the gun up and away himself.
Allegedly, the soldiers left Miller's home in shambles. In some versions of the story, even the gold dentures were stolen from his night stand. All of his most important financial papers were protected.
In his will, Miller bequeathed both money and land to the city of Lynchburg — including the land that would become Miller Park. His educational endowments today help support the University of Virginia and two institutions that bear his name: The Miller School of Albemarle and the Miller Home for Girls of Lynchburg.
Walking through the front yard, Julie and Richard are trailed by Sophie, a curious orange and white cat, while rescue dogs Archie and Stella pant behind a wooden fence. They find things in the yard sometimes; not treasure, yet, but tools and fragments, old glass bottles or a metal spade.
"Things float up," Richard said. It's a unique, historic property, but one that feels like theirs. Standing in the yard, hands in his pockets, Richard looks up at one of the attic dormer windows.
"At 9 or 10 o'clock, I always look up there thinking somebody is going to be looking back down."
Despite her best efforts, Julie said they haven't experienced any "occult phenomena," no ghost of Samuel Miller floating through the halls. "I'm waiting," she said, laughing.
Richard said the home is most beautiful when the sun is out. Gesturing to Julie, he said it's the "first house I've built into a home, and it's because of her."