‘You Must Fall Back’ – Butler Confounded at Bermuda Hundred

May 16, 1864 (Monday)

Butler's Headquarters at Bermuda Hundred
Butler’s Headquarters at Bermuda Hundred

While General Grant threw the Army of the Potomac south toward General Lee’s Army of Northern Virginia, two other armies were in motion. In the Shenandoah Valley was Franz Sigel and his unnamed army from the Department of West Virginia. Up the Virginia Peninsula at Bermuda Hundred was Benjamin Butler’s Army of the James, poised to threaten Richmond.

According to Grant’s original strategy, “Butler was to advance by the James River, having Richmond and Petersburg as his objective.” He was not to capture either, but rather to sever the railway linking the two cities so that the Confederates in Petersburg, under P.G.T. Beauregard, could not reinforce General Lee.

With over 30,000 men dispersed into two corps, Butler was to move up the Peninsula, steaming up the James River, landing the troops at the village of Bermuda Hundred, near the mouth of the Appomattox River.

Map of the landing, Petersburg and Richmond.
Map of the landing, Petersburg and Richmond.

And with perfect timing, this was accomplished. Butler landed his men on May 5, the same day Grant engaged Lee in The Wilderness. But this was when Butler wanted to change the plan completely. Through the word of a spy, Butler learned that Richmond was scantily defended, that through a rush and a push, it would be his. While Eighteenth Corps commander, General “Baldy” Smith, didn’t care for the idea, General Quincy Gillmore, commanding the Tenth Corps, openly refused to follow the order if Butler issued it.

Butler backed off on the idea and began to throw up a series of entrenchments, their left on the Appomattox River, their right on the James. In the meanwhile, Federal gunboats chuffed up the latter river, probing for infantry to be seen from the shore as a sort of water cavalry. Butler had also dispatched actual cavalry on a raid from the Norfolk area. They would play hell along the railroad to Petersburg, as well as the rail from Petersburg south, before rejoining Butler’s main army on the 10th.

By that time, things had quickly gone sour. The navy lost two vessels to torpedoes and now refused to assist Butler at all. On the 6th, General Gillmore refused Butler’s order to send a brigade toward Port Walthall Junction, a small town a few miles from Butler’s left flank that might be the key to slipping around the Confederate defenses.

Open this map in a new window and use it while you read!
Open this map in a new window and use it while you read!

The Confederate defenses were hardly that. General Beauregard was not yet on the scene, leaving Petersburg in the questionable hands of General George Pickett. He had called for reinforcements, but as Southern troops passed through Petersburg, he did not have the authority to stop them. All he could throw at the Federals for the time being was 600 men. And throw them, he did.

Stepping out from the Federal lines, a single brigade under General Charles Heckman advanced four miles and was turned back by these 600. It was a timid affair, with neither side getting a good look at the other, and both sides believing they faced more men than was actually so. Heckman returned, and “Baldy” Smith bemoaned Quincy Gillmore.

The next morning (the 7th), the Rebels were truly reinforced, gathering upwards of 2,600 men. But Butler now hurled more than a brigade. Two Rebel brigades under Johnson Hagood were approached by 8,000 Yankees arrayed. After an initial probe and encounter with Federal cavalry, Hagood’s Confederates retreated to their defenses along the railroad connecting Petersburg to Richmond. The Federals attacked first their left flank, reaching the railroad, but bogged down by the terrain and Rebel fire. In the end, the railroad was mostly protected and Pickett was afforded another day to gather troops. But fearing for their safety, he ordered Hagood’s men to abandon their positions and move south to the other side of Swift Creek.

While this protected Petersburg, it left the railroad open for Butler’s men to ply their trade. For this, Butler selected seven brigades, moving them from his defenses on the 9th as if to stab toward Petersburg. After a brief skirmish, however, he did not do too much testing of the Swift Creek defenses and seemed content enough with ripping up track to Richmond.

Union works at Bermuda Hundred.
Union works at Bermuda Hundred.

As Butler’s main advance was south-facing, from their rear at Drewry’s Bluff came a Confederate division under Robert Ransom. On the 10th, not knowing of Butler’s shift south toward the Swift, Ransom’s men stumbled upon an exposed Federal regiment at Chester Station, several miles behind Union lines. They attacked, and the regiment sent word back to Butler.

Having received word from Washington that Grant’s campaign was a smashing success, he decided it best to withdraw from the southern-facing Swift Creek line and return to his original westward-facing lines to then threaten Richmond. The idea had always been to link up with Grant’s army as soon as they appeared outside the gates of Richmond.

But the distance was not far, and Gillmore raced his Tenth Corps to meet the new threat. For a time, it appeared as if Ransom’s Rebels would have the day, but with the arrival of the first brigade from Gillmore’s Corps, they were thrown back in confusion. The Federals returned to their lines at Bermuda Hundred and Butler waited for his next idea.

Moving on the 12th, Butler’s two corps marched along parallel roads first west then then north upon reaching the railroad. On the 14th, they launched their attack with great initial success, capturing a line of Rebel works along the Petersburg Turnpike and Woodridge Hill. With this, Ransom’s Confederates withdrew to their second line of defenses. These, however, could not be taken. That night, the Rebels received more reinforcements and Butler swapped out his spent regiments with fresh ones. The next morning, he wanted to attack.


But Baldy Smith heavily cautioned against it, and before long, Butler agreed, and placed his entire army on the defensive where they stood, across the Petersburg & Richmond Railroad. Interestingly enough, P.G.T. Beauregard, now reinforced, was also planning an attack. His was to be launched on the 18th, but Richmond thought that too long to wait and see what Butler might do next. It was moved up to the 16th – the morning of this date.

His was a grand scheme, involving troops from Petersburg, now under General W.H.C Whiting, to attack the Federal rear as his troops from Drewry’s Bluff assailed their front, both attacking the army between them. This trap would ensnare the Federals and throw them back to at least their Bermuda Hundred defenses, reopening the railroad in the process.

Before the dawn (on this date), all was set into motion. Ransom’s Confederates struck the Union right, as Whiting’s Rebels followed orders to march toward Port Walthall Junction, along the railroad. From there, they were to follow the sound of the guns.

Ransom’s attack fell upon Baldy Smith’s men, but it was soon turned back and bloodied. The Federals, too, suffered, being shifted and bent on the right. To all, it appeared to be breaking. Butler had ordered Gillmore on the left to attack at 6am, but that time was long past, and both sides paused. Butler waited for Gillmore, as Ransom waited for Whiting’s troops from Petersburg.

Gillmore did not receive Butler’s order until 6:20am, pondered it for a bit, and then planned to follow it. Before he could, however, Confederate skirmishers appeared before his lines and he assumed he was about to be attacked. Following a flurry of dispatched between Gillmore and Butler (in which Gillmore claimed to have been assaulted three times), the former was finally given permission to use his discretion. Gillmore gladly elected not to attack, but shortly after, Butler decided to shift his entire line to the right, and Gillmore was to conform.

Instead, Gillmore now decided to attack. He figured that if the Rebels were counterattacking on the right, he would hit them in their flank. While half of Gillmore’s men found this transition difficult, the other half found it impossible, as the Rebels in their front finally launched an organized attack. While all this was going wrong, Butler decided to completely withdraw Baldy Smith’s Corps on the right. Gillmore was to act as a plow, while Smith maintained some sort of rear guard.

Butler's eventual signal tower.
Butler’s eventual signal tower.

To the south at Port Whithall Junction, General Whitings Confederates had stopped, and could see before them a few Union regiments under the command of Adelbert Ames. Whiting pursued them for a time, but quickly became convinced that there were Yankees everywhere. Eventually, he would order a general withdrawal, but not before his presence caused a stir in Butler’s lines.

“You must fall back,” came the message from Butler, “press to right, and get in rear of Smith’s corps. He will try and hold his ground until you get in his rear and clear the road to the intrenchments, so that we may get behind the defenses. Push vigorously.”

From the sound of the dispatched, the Rebels had somehow managed to get in the Federal rear. This was all sort kind of confusion, and Butler had no confirmation of their presence. It was reported, however, that even more Confederates were crossing the James River in an attempt to cut Butler off from his original defenses. It was all an immense confusion, and Gillmore simply decided to storm south and see what might happen.

By 2pm, Butler had managed to reestablish his lines a mile or so to the rear at the Half Way House. Now secure, he ordered Baldy Smith’s Tenth Corps to advance and retrieve as many wounded as they could. Two hours later, Smith concluded that the Rebels in their front were too strong even for that, and Butler finally gave the order to return to the original entrenchments And by dark, they were back at Bermuda Hundred.

Soon, Beauregard’s Confederates would construct their own lines opposite Butler’s and effectively bottle him up. The whole affair was rather pointless, and hardly even a distraction.1

  1. Sources: The Bermuda Hundred Campaign by Bruce R. Wells; Back Door to Richmond by William Glenn Robertson; Life of Benjamin F. Butler by Thomas Augustus Bland; The Military Operations of General Beauregard by Alfred Roman. []
Creative Commons License
‘You Must Fall Back’ – Butler Confounded at Bermuda Hundred by CW DG is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International


View all posts by