The RSA Conference 2024 is upon us, and this year promises to be a treasure trove of insights for security professionals. We're particularly excited to explore the latest advancements in areas like AI-powered security and cloud protection. To enhance our analysis, we're putting Google Gemini, a large language model, to the test. Can it navigate the vast amount of information available about the conference and identify the themes that matter most? In this blog, we’ll explore the conference through the lens of Google Gemini, offering previews of sponsor briefings, session topics, and emerging trends that will shape the future of cybersecurity, and we’ll also examine Gemini’s ability to analyze text data.
What does Gemini already know about the conference?
One of the first questions we asked Gemini was what it knows about the RSA conference for 2024.
“The RSA Conference 2024 is just around the corner, happening from May 6th to May 9th in San Francisco at the Moscone Center. The theme this year is ‘The Art of Possible,’ focusing on innovation and creativity in cybersecurity,” was the response, with links to the RSA conference registration portal and discussion of keynotes, expo hall, networking opportunities, and the Innovation Showcase.
This analysis is spot-on, and Gemini successfully provided relevant information about the conference. We decided to dive a little deeper and started asking it specific information, such as details on the first keynote. While Gemini successfully informed us that the first keynote is on May 6th at 3PM and is titled “The Power of Community,” it began to struggle with additional details and informed us that “the speaker list for the opening keynote hasn’t been revealed yet.” Of course, the RSA conference website clearly illustrates that Hugh Thompson, Executive Charman of RSAC, will be the presenter. We were willing to chalk this up to outdated information, so we continued the experiment.
Feeding Gemini the keynotes
Next, we asked Gemini for a list of 2024 keynotes. Gemini once again stated that it had incomplete data, but provided a list of “confirmed keynote speakers.” The list was accurate and included six of the keynote speakers. At this point, we decided to “fill in the blanks” for Gemini and provide information on the keynotes.
After providing a list of keynote titles and speakers to Gemini, it provided the following analysis:
- There's a strong focus on artificial intelligence (AI) this year, with multiple keynotes exploring its security implications, ethical considerations, and potential benefits.
- Several sessions feature high-profile government officials and industry leaders, including Jen Easterly (Director of CISA), Chris Krebs (Chief Intelligence Officer at SentinelOne), and Anne Neuberger (Deputy National Security Advisor).
- There are also some lighter keynotes, like the conversation with actor Jason Sudeikis and the closing performance by Alicia Keys.
Once again, Gemini was fairly accurate in its analysis. We asked Gemini to build a table of topic keywords for the keynotes. Gemini built the table, but included a disclaimer that the information was estimates only. Using this table, we took the top 10 and built a table of topics from the titles.
While Gemini did successfully analyze common themes at the conference, its counts were in only a few cases significantly off. The most notable differences are “Government” and “CISO” keywords. However, when diving into the data further, we can better understand why these counts are off.
For government, Gemini didn’t focus on just the keyword “government” but also references to government, such as references of federal agencies. In fact, if you look at instances of the word “national” instead of “government,” you’ll notice that Gemini’s estimate was only off by one. This means Gemini isn’t just counting words, it’s understanding the context of those words and grouping them into actual topic categories.
CISO, the other data discrepancy, is explained by looking at the session “CISOs Unchained,” which has the word CISO in its description six times very close together. Keep in mind that Google built Gemini, so it’s likely Gemini uses some of Google’s search indexing technology for “counting” keywords. In this case, because CISO is overused in the description, Google’s indexing would only count CISO once, instead of six times. This reduces the discrepancy significantly.
Learning from the Learning Labs
We applied the same methodology to Learning Labs and experienced similar results.
Once again, while Gemini’s counts were not completely accurate, they were accurate enough to analyze trends in the data.
Struggling with Sponsor Briefing Analysis
Finally, we decided to put Gemini to the test by asking it to analyze 68 Sponsor Briefings for themes, as well as identify the sponsors. For some reason, Gemini struggled with this task as written and appears to have significantly hallucinated some of the data. Note that we asked Gemini to only evaluate the titles of the sessions, not the actual descriptions of the sessions. The results are that it appears Gemini may have hallucinated the session descriptions based upon the titles, then utilized this data to provide its response.
What’s interesting is that, except for “data,” Gemini’s hallucinations were rather accurate in determining the content of the briefings just based upon the title. To be clear, we did not request Gemini to infer this data. It chose to do this on its own because it believed that was the data we needed.
We also asked Gemini to produce a list of sponsors from the briefing list. Unfortunately, Gemini became slightly confused with some of the non-standard briefing names and started listing some of the sponsors as “How Hub International achieved cyber resilience with Cohesity” and “Shine a light on shadow IT.” However, for the standardized briefing names, which contained the vendor name followed by the topic, Gemini was able to successfully parse the vendor names from the title.
Conclusions and What to Expect
While Gemini did reasonably provide insights into trends for the RSA Conference, it struggled with ensuring complete data accuracy, especially with exact word or topic counts. However, Gemini did successfully provide textual analysis of the conference themes and topics, so that shows promise. With that said, we’ll conclude this experiment with Gemini’s own words on what to expect for the RSA Conference 2024.
“This year's theme, ‘The Art of Possible,’ reflects the industry's focus on innovation and creative solutions. Expect to see leading cybersecurity companies showcasing their latest advancements in areas like AI-powered security, cloud protection, and zero trust architectures. The conference will also feature insightful keynote addresses, informative sessions, and a bustling expo hall where attendees can network with industry experts and discover cutting-edge solutions. Whether you're a seasoned security professional or just starting out, RSA 2024 promises to be an essential event to stay ahead of the ever-evolving cyber threat landscape.”
