The officer had been accompanying the foreign secretary on a trip to the US.
A Metropolitan Police spokesperson said: "We are aware of the incident on a flight into the UK on Friday, 18 September and we are taking this matter extremely seriously.
"The officer involved has since been removed from operational duties whilst an internal investigation into the circumstances is taking place."
The weapon was reportedly found by a cleaner.
It follows a similar incident in February of this year, when David Cameron’s bodyguard was removed from operational duties after leaving his gun in a toilet on a transatlantic plane.
The gun, believed to be a 9mm Glock 17 pistol, was handed into flight attendants by a passenger.
As a former prime minister, Mr Cameron is entitled to continued security by the Metropolitan Police.
The foreign secretary travelled to Washington to discuss the issues of Brexit and Northern Ireland with senior US politicians, including Mike Pompeo, the US Secretary of State.
Mr Raab said he had had "very positive discussions" with Republicans and Democrats after senior US figures warned Britain must uphold its commitment to Northern Ireland as part of Brexit.
Nancy Pelosi, the Democratic Speaker of the US House of Representatives, said that while she "welcomed his assurances", breaking its international agreements will have consequences for future trade deals with the US.