They are generative AI models against hackers in DEF CON’s AI Village

One of the most effective ways to test the security of an application is through the use of adversarial attacks. In this method, security researchers actively attack technology, in a controlled environment, to try to find previously unknown vulnerabilities.

It’s an approach that is now being championed by the Biden-Harris administration to help secure generative artificial intelligence (AI). As part of its Actions to Promote Responsible AI announcement yesterday, the administration called for public assessments of existing generative AI systems. As a result, this year’s DEF CON 31 security conference, which runs from August 10-13, will feature a public evaluation of generative AI at the AI village.

“This independent exercise will provide critical information to researchers and the public about the impacts of these models, and will allow AI companies and developers to take steps to fix problems found in those models,” the White House said in a statement. release.

Some of the leading vendors in the generative AI space will be participating in the AI ​​Village hack, including: anthropicGoogle, hug faceMicrosoft, Nvidia, open AI and AI stability.

DEF CON villages have a history of advancing security awareness

The DEF CON security conference is one of the largest gatherings of security researchers in any given year and has long been a place where new vulnerabilities have been discovered and disclosed.

This won’t be the first time a town at DEF CON will take aim at a technology that’s making national headlines. In previous years, especially after the 2016 US election and fears about election interference, a Voting Village was established at DEF CON in an effort to look at security (or lack thereof) of technologies, infrastructure, and assets. voting machine processes.

With the villages at DEF CON, attendees can discuss and research technologies in a responsible disclosure model that aims to help improve the overall security state. With AI, there is a particular need to examine the risks of the technology as it becomes more widely deployed in the broader society.

How the generative AI hack will work

Sven Cattell, the founder of AI Village, commented in a statement that, traditionally, companies have solved the problem of risk identification using specialized red teams.

A red team is a type of cybersecurity group that simulates attacks in an effort to detect potential problems. The challenge with generative AI, according to Cattell, is that much of the work around generative AI has been done in private, without the benefit of a red team evaluation.

“The various problems with these models will not be resolved until more people know how to red-team and test them,” Cattell said.

Specifically, the AI ​​Village generative AI attack simulation will consist of on-site access to extensive language models (LLMs) from participating vendors. The event will have a capture the flag point system approach in which attackers earn points for achieving certain objectives that will demonstrate a variety of potentially harmful activities. The person with the most points will win a “high-end Nvidia GPU”.

The evaluation platform in which the event will take place is being developed by AI scales. “As basic models become more widely used, it is critical to ensure they are carefully tested for reliability and accuracy,” Alexandr Wang, founder and CEO of Scale, told VentureBeat.

Wang noted that Scale has spent more than seven years building AI systems from the ground up. He claims that his company is also impartial and not committed to any particular ecosystem. As such, Wang said Scale can independently test and evaluate systems to ensure they are ready to deploy to production.

“By bringing our experience to a broader audience at DEF CON, we hope to ensure that progress in base model capabilities comes along with progress in model testing and safety,” Wang said.