The permanence of information on the Internet, in some cases, goes against the basic principle of fairness in America, Google's executive chairman, Eric Schmidt, said during a Monday talk at New York University.

Schmidt gave an example to illustrate his point: a teenager commits a crime, goes to juvenile hall, and is released. In America's legal system, the person can apply and have the crime expunged from their record then legally state they were never convicted of anything.

But the same isn't true on the Internet. A potential employer could still run a reference check online and find out about a crime someone committed years ago as a dumb teenager.

"That seems to violate our innate sense of fairness," Schmidt said. "This lack of a delete button on the Internet is a significant issue, when you frame it in that context." He added that there are times when erasure is the right thing to do, legally, and other times when would be inappropriate.

"How do we decide?" he questioned. "We have to have that debate now, in America."

Lightening the mood, Schmidt jokingly proposed that "at the age of 18, you should just, as policy, change your name."

Schmidt was at NYU with Jared Cohen, the director of Google Ideas, to discuss their new book, The New Digital Age: Reshaping the Future of People, Nations and Businesses. The wide-ranging discussion touched on everything from privacy concerns in today's rapidly evolving digital age to the uncertain rules of cyberwar.

On that front, Schmidt acknowledged that researchers are concerned about attacks against the country's electric grid, water systems, and financial trading computers. But he said that critical infrastructure owners and operators are working hard to ensure their mission-critical systems are protected and firewalled from the public Internet.

Sophisticated cyber attacks against everyday businesses are a more likely problem, he said. Chinese hackers are aiming to penetrate companies' networks and siphon off information – and all businesses are a target.

"It's probably the case that most companies have [hackers] inside of them today," he said, adding that the problem is "only going to get worse."

At this point, the relationship between the U.S. and China in cyberspace is "just about as adversarial as you can imagine," Cohen added.  

"The real question to ask is – at what point [is a] cyberattack so severe and destructive that it warrants a physical world response?" he said. "We don't know what that limit is yet."