The Civil War had a profound impact on the lives of the people along the C&O Canal. Some may say the C&O Canal may have been the true dividing line between the North and the South. A Potomac crossing by either side would necessarily have included a Canal crossing.

  Important river crossings near Poolesville, Md. were Conrad’s Ferry and White’s Ford. On a hillside overlooking these areas stands a beautiful manor home now called Annington. It has breathtaking view of the Potomac River, the C&O Canal and the rolling countryside of Maryland and Virginia. From the porch one can even see the prominent peak named Balls Bluff near Leesburg, Va. This strategic attribute would ensure the homes’ occupation by Union forces through the entire War. It became a natural place for military officers to make plans even as they stood lookout from the grounds.

  Early in the Civil War, on the night of October 20, 1861, Union forces began crossing the Canal and Potomac to intercept Confederate troops spotted by a reconnaissance patrol near Balls Bluff. Even a Canal barge was enlisted into the operation. Colonel Edward D. Baker, a U.S. Senator from Oregon and former law partner of his friend, President Abraham Lincoln, was dining that same evening at Annington. He was no doubt present as the troops were being deployed. At the dinner, he was reported to have toasted, “Tomorrow night I will dine either in Leesburg or in Hell.” And then, by tradition, he and fellow officers threw their glasses into the fireplace.

  Very early the next morning, a series of mishaps and miscommunications would draw over 1700 troops across the river into conflict with an equal size Confederate force. Union troops would engage the enemy at Balls Bluff led by Colonel Baker and opposed by Colonel E. V. White; the very man that White’s Ford was named for. Colonel White grew up here and knew the area intimately and would deftly out-maneuver his rival from Oregon. The Northern troops would be trapped in a mostly open area with their backs to the bluffs and in rough terrain. Fighting, sometimes hand-to-hand, raged throughout the day. Cannon fire was exchanged, confusion reigned, advances and retreats marked the afternoon. The smoke, the noise, the screaming. One Lieutenant, Oliver Wendell Holmes, Jr. would survive the day badly wounded. His future would guide him onto the U.S. Supreme Court. Sadly, Colonel Baker’s fate was not so glorious. He himself would fall between 4:30 and 5:00 p.m., becoming the only U.S. Senator to die in battle.

  Just before dark, Colonel Baker’s replacement attempted to break through Confederate lines. The ploy failed and a general panic ensued. Many men died or were injured fleeing down the cliffs. Many more were picked off by the Rebel troops firing down from above. Some would drown trying to flee across the Potomac. The Canal barge was swamped by the desperate men. Union losses totaled 49 dead, 158 wounded, 553 captured and 161 missing. News of the shocking defeat would reach Washington at the same time the bodies floating down the Potomac River would.  

  Congress convened a special committee to study the disaster, pinning blame on Colonel Baker’s commanding officer, General Stone. However, some have put the blame squarely on Colonel Baker’s shoulders. Either way, Baker’s funeral was held at the White House, President Lincoln would mourn the loss of a good friend and Annington would gain a ghost.

  The haunting of Annington has been reported for some time. Workmen have told of hearing crying in the supposedly empty house. The sound of glass breaking in the fireplace has been heard. One recent guest at the house, having trouble sleeping, tried the sofa and awoke to a force like a heavy weight pinning him down. Other manifestations, the feeling of a sad presence, a chair moved while a back was turned; these things all add to the legend.

  Perhaps Colonel Baker is still attempting to right his wrongs, win the battle, fighting again in his soul and with eternity. Something is keeping him tethered to this world. Maybe his spirit wanders down to the Canal and the ferry, trying to cross and rejoin the battle. After the War, the successful businessman E.V. White bought Conrad’s Ferry. It was renamed and operates still to this day as White’s Ferry. Perhaps it is old E.V. himself who bars the way and won’t let the Colonel cross to the other side.