Google and Verizon suggest treating fixed line services differently to wireless and some specialised content.
This would allow net providers to give priority to certain online traffic.
Protesters outside the famed Googleplex said this would create a “pay-to-play” service and urged Google to live up to its famous motto “don’t be evil”.
“Companies like Google have benefited from a free and open internet and their plan will destroy that,” said James Rucker of ColorofChange.org, one of many consumer and public advocacy groups taking part in the event.
“They are talking about producing a fast lane, essentially a higher tier, for premium content that means if you want to play in the 21st Century internet you will have to pay.”
The proposals unveiled this week by the search giant and telecom titan Verizon champion an open net for wireline services but suggest loopholes for wireless and what they called “differentiated” content.
Critics have said this would undermine the principle of net neutrality where all web data is treated equally and no-one is given preferential treatment or discriminated against.
“Whether you are a blogger, an entrepreneur, a journalist or someone trying to organise a community, the internet is precious,” said Mr Rucker.
“We all want to stand together to ensure it is protected for the future. We would expect Google to take leadership in making that happen, not be on the front line of undoing that.”
Google and Verizon made their announcement after the Federal Communications Commission (FCC) ended closed-door talks with service providers and internet companies to find a consensus on the principle of net neutrality.
The FCC is trying to navigate what it has called a “third way” to resolve the issue after its authority was called into question when a court ruled it had no power to sanction Comcast for slowing some net traffic.
Net neutrality is seen as central to the government’s broadband plan to provide high speed access to every citizen by 2020.
Protestor Christine Springer criticised the lack of leadership coming from the agency.
“The FCC is sitting on their hands. They are hoping nobody will notice but unless we make a lot of noise the corporate giants will prevail. The job of the FCC is to regulate not negotiate with giant corporations.”
Those taking part in the rally agree and chanted slogans like “net neutrality is under attack, stand up and fight back” and “we demand our internet rights, together we stand together we fight”.
There was also some singing to the tune of “Clementine” organised by a group of senior citizens calling themselves the Raging Grannies.
“We want to raise awareness about this issue and shine a light on how important it is to keep the internet free and open to one and all,” said Raging Granny Gail Sredanovic.
Martha Champion donned a heavy black Victorian costume to drive home her concerns.
“I am in mourning for the death of the internet and believe this plan will lock out those that can’t afford to pay a premium for their content to load faster or for their site to go quickly.”
The rally also attracted the very young. Seven year old Alexis Buggs said she took part “to help save the internet”.
Her mum Erin Hodgson told BBC News “I’m a stay at home mom and so it was either go to the park or come out here and take a stand and teach my kids about putting our voice out there and being proud to be American.”
The rally organisers presented Google with boxes of petitions they claimed held the signatures of 300,000 people opposed to anything that would harm the principle of net neutrality.
guy carrying a box of petitions with Google don’t be evil sign The petition signatures were collected over the last couple of weeks
Google asked those that took part in the protest to fill out a form and submit their own comments about the proposals.
Afterwards the company’s head of public policy, Nicklas Lundblad spoke to reporters.
“This is an important issue, a complex issue and it deserves to be discussed. Google is a fierce supporter of an open internet and we see that we have a couple of key enforceable protections in our proposal with Verizon and that is much better than no protections at all.
“This issue has been at a standstill for quite some time and we think this proposal is a way to advance that discussion.”
In a move that comes as no surprise, telecom company AT&T has given its backing to the plan while firms like Facebook and Skype have denounced it.