Prince Harry has filed a claim for a judicial review over a Home Office decision to prevent him from paying for police protection for himself and his family during a visit to the UK.
A legal representative for Harry said he wants to bring his son Archie and seven-month-old baby daughter Lilibet to visit from the US, but fears they will be "unable to return to his home" because it is too dangerous.
The duke wants to fund the security himself, rather than ask taxpayers to foot the bill after he stepped down as a senior member of the Royal Family.
Harry is now arguing his private protection team in the US does not have access to UK intelligence information which is needed to keep his wife and children safe.
The duke's legal representative said in a statement: "The UK will always be Prince Harry's home and a country he wants his wife and children to be safe in.
"With the lack of police protection comes too great a personal risk."
The representative added: "The Duke and Duchess of Sussex personally fund a private security team for their family, yet that security cannot replicate the necessary police protection needed whilst in the UK.
"In the absence of such protection, Prince Harry and his family are unable to return to his home."
Harry briefly returned from the US last July for the unveiling of a statue of his late mother Princess Diana.
The day before, he met seriously ill children and young people at a WellChild garden party and afternoon tea in Kew Gardens, west London.
However, it is understood the duke's car was chased by photographers as he left.
Harry has shared his concerns in the past about his family's safety.
Diana died in a car crash after she was chased by the paparazzi in Paris in 1997.
The legal representative added: "Prince Harry inherited a security risk at birth, for life. He remains sixth in line to the throne, served two tours of combat duty in Afghanistan, and in recent years his family has been subjected to well-documented neo-Nazi and extremist threats.
"While his role within the institution has changed, his profile as a member of the Royal Family has not. Nor has the threat to him and his family."