Speaking in a new interview for the Color of Change initiative, Prince Harry said tackling the issue was "not down to the black community".
In a clip of the interview with US civil rights leader Rashad Robinson, Harry also praised young people for "acting, rather than just saying" over discrimination.
The 35-year-old - who stepped down as a senior royal earlier this year along with his wife, the Duchess of Sussex, Meghan Markle - added that it was "down to every single person that is on the planet right now" to end racism.
"We have to go to the root of the problem, and actually fix it there," he said.
The clip of the interview was shared over the weekend online, with the caption seemingly including a quote from Prince Harry: ''Inequality is not unfortunate like a car accident, it has been manufactured."
It continued: "Prince Harry and @iamrashadrobinson talked candidly about systemic racism, hate online and people with privilege, using their privilege for change.
"Here’s a sneak peek at what happened. Stay tuned for the full convo #ComingSoon!"
In the clip, Mr Rashard said: "Part of this work, part of this moment that we're in, is that making sure from analysis to action that we drive action toward structural and systemic change.
"Changing the rules that have held people back."
Harry added: "You say this so beautifully Rashard, and it's so true. There's so much that has happened for so long and once those problems exist, charity is crucial in so many ways but it can't fix the problem.
"The problems already exist, and are too big. As I've said to you before, it's like bringing a bucket of water to a forest fire."
Harry's comments came as the authors of his and Meghan's upcoming biography Finding Freedom claimed the latter had helped guide the Duke on his “journey to wokeness".
Co-author of Finding Freedom Omid Scobie told US media outlet National Public Radio: “Harry’s journey to wokeness has been very public.
"We've seen him learning and educating himself along the way, but this experience of witnessing Meghan face racist remarks and commentary would have been the first time he'd seen someone in his life or someone he was particularly close to affected by it in a certain way.”
Meghan has repeatedly been targeted by racist trolls on social media in the past, while some have pointed at the double standard regarding how Meghan is reported on in headlines in contrast to white female members of the Royal Family.
Early on in their relationship, Harry issued a statement through royal channels in which he said there had been a “line crossed” - calling out the “racial undertones of comment pieces, and the sexism and racism of social media trolls and web article comments".
The couple are currently living in Los Angeles with their son Archie Harrison Mountbatten-Windsor after stepping down as senior royals on March 31.