His doing so drew widespread criticism from former and current military leaders.

The nation’s top military official has apologized for taking part in President Donald Trump‘s walk from the White House to St. John’s Church for what eventually turned into a controversial photo op after authorities had used pepper balls and smoke cannisters to disperse largely peaceful protesters.

“I should not have been there,” said Gen. Mark Milley, the chairman of the Joint Chiefs of Staff, in a prerecorded video commencement address to National Defense University. “My presence in that moment and in that environment created a perception of the military involved in domestic politics.”

“As a commissioned uniformed officer, it was a mistake that I have learned from,” Milley said in the video address.

Milley acknowledged in his remarks that everything senior leaders do “will be closely watched and I am not immune, as many of you saw, the result of the photograph of me at Lafayette Square last week that sparked a national debate about the role of the military in civil society.”

Milley, who was wearing combat fatigues, and Defense Secretary Mark Esper were the target of widespread criticism from current and former leaders for their participation in last Monday’s walk through the park to St. John’s Church. A senior U.S. official said Milley had changed into the camouflage uniform from his dress uniform in anticipation of a long evening at the command center that had been set up in Washington to monitor Monday evening’s protests in the capital.

Both believed they were accompanying President Trump to thank National Guard troops and other law enforcement officers outside Lafayette Square, Esper said last week.

When Trump’s party arrived at St. John’s Church, President Trump posed with a Bible and then asked top officials, including Esper, to participate in a photo opportunity. Though he was nearby Milley was not asked by Trump to join in the picture.

Earlier on that Monday, Esper and Milley had pushed back strongly against President Trump’s demand that thousands of active duty troops be sent to the nation’s capital under the Insurrection Act to quell the protests that emerged following the death of George Floyd in Minneapolis, according to a senior U.S. official.

Both succeeded in advocating that thousands more National Guardsmen from other states should be sent to reinforce the numbers of the D.C. National Guard in a show of force that would demonstrate that active duty troops were not needed. As a back-up 1,600 active duty troops were moved to areas outside of Washington D.C. to be on standby in case they were ever needed, but tthey were sent home a few days later.

In his commencement address, Milley said he, like many Americans, was “outraged by the senseless and brutal killing of George Floyd.”

“His death amplified the pain, the frustration, and the fear that so many of our fellow Americans live with day in, day out,” Milley said.

“The protests that have ensued not only speak to his killing, but also to the centuries of injustice toward African Americans,” he added. “We should all be proud that the vast majority of protests have been peaceful.”

“What we are seeing is the long shadow of the original sin in Jamestown 401 years ago,” said Milley. “We are still struggling with racism and we have much work to do.”

And that includes in the military where Milley said “we must, we can, and we will do better.” The senior leaders of the four military services have also said that they will lead initiatives to discuss and improve race relations within the military in the wake of Floyd’s death.

Earlier this week Milley and other top Pentagon officials indicated they were open to the idea of having discussions about the renaming of 10 Army bases and facilities named after Confederate generals.

But President Trump quashed that possibility on Wednesday tweeting that “My Administration will not even consider the renaming of these Magnificent and Fabled Military Installations, Our history as the Greatest Nation in the World will not be tampered with. Respect our Military!”