Groundbreaking Innovations from Siggraph 2024

What is Siggraph?

Special Interest Group on Computer Graphics and Interactive Techniques, Siggraph is an annual conference that celebrated it’s 50th anniversary by returning to Denver this year. Historically it’s a bit of a geek fest, with computer scientists and pHDs coming together to share papers and university researchers showcasing their latest work.

NVIDIA has been a huge sponsor of Siggraph over the years. Its founder, Jensen Huang took over the main ballroom for a fireside chat with Mark Zuckerberg. This gave the event an air of celebrity this year, and shifted the dialogue from computer graphics to artificial intelligence.

AI was enabled by GPU-accelerated computing, a technology pioneered by NVIDIA in the early 2000s, transforming the landscape of computer graphics, and opening up the ability to do scale computing power exponentially, leading to where we are today.

Generative AI

Generative AI is the hype word of the year. At Siggraph 2024, this technology was all over the conference, demonstrating its potential impact on visual storytelling and immersive experiences. NVIDIA isn’t just building GPUs anymore, they’re building massive data centers to unleash a suite of services that is driving their valuation into the trillions of dollars.

I didn’t see that coming btw, but I am not the only one. During the fireside chat, Meta CEO Mark Zuckerberg admitted he thought we would have holographic displays in smart glasses way before we had AI voice assistants.

Despite it’s current limitations, six fingered hands and hallucinations, AI’s ability to create lifelike graphics and complex simulations was showcased by dozens of companies. These advancements include:

• Hyper-realistic Textures: New algorithms enable the creation of textures that closely mimic real-world materials. HP even showed a texture scanner that can convert almost any object’s surface into a texture applied to a 3D object.
• Dynamic Environments: AI-driven systems can now generate entire ecosystems that evolve based on user interaction. Check out this 1 minute video from WPP, Shuttestock and NVIDIA
• Markerless MoCap: New camera based technology that allow for more fluid and natural character movements without wearables or tracking markers.

Insights from Industry Leaders

During fireside chats, Jensen Huang (NVIDIA) and Mark Zuckerberg (Meta) shared their perspectives on the future of AI. Jensen talked about Software 3.0, which is the term being used where humans and machines working closely together to create intelligent software development. Basically humans prompt AI to generate useable code. He said every NVIDIA engineer works with an intelligent agent to augment their coding ability and efficiency.

Mark Zuckerberg discussed how software 3.0 will revolutionize content creation. He highlighted that “everyone will have an AI agent working with them side by side,” enabling creators to focus on higher-level tasks while AI handles intricate details. Zuckerberg suggested that soon you’ll be able to even kickoff weeks or months-long workflows that are entirely managed by various AIs.

Can AI Teams Build Virtual Words?

Both executives talked about how teams of function specific AIs will work together, as teams, with a management AI overseeing their cooperation. AIs work with industry standard outputs to make handing off work in progress from one model to another possible.

For example, USD (Universal Scene Description) is an open standard language for the 3D internet. It describes geometries, materials, physics, and behavior representations of 3D worlds. USD allows 3D artists, designers, developers, and others to work collaboratively across different workflows and applications as they build virtual worlds

NVIDIA’s Text to USD service can automatically generate code from text prompts. This technology allows users to convert text descriptions into fully realized 3D scenes using generative AI. The process involves:

1. Text Input: Users describe their desired scene in natural language.
2. USD Generation: The system translates this description into a USD file, creating a digital blueprint.
3. AI Refinement: Generative AI then enhances this blueprint, adding realistic textures, lighting, and animations.

Teams of smaller AI models with specific job functions manage different aspects of this pipeline, overseen by a central AI. This hierarchical structure resembles an organizational chart, ensuring efficient and cohesive output.

The example they showed was:

1. Using text to 2D image AI to create a simple shape as a jpeg.
2. Moving the 2D image into a 3D AI generator with output as USD.
3. Uploading the 3D USD into an LLM to add color, texture, and background context.

Virtual Agents: The Interface to AI

Jensen promised that virtual agents, powered by large language models, will transform customer service by providing personalized, efficient interactions. They’ll be taught on company specific data like operating manuals, FAQs, product design documents, etc. These agents use advanced algorithms to understand and respond to user queries in real-time, offering a seamless experience that closely resembles human interaction.

Zuckerberg believes that their CODEC avatars are close to release. These lightweight realistic virtual humans will enhance user experience, making AI interfaces more relatable and trustworthy. This will not only improve customer satisfaction but also opens up new possibilities for virtual interactions in sectors like healthcare, education, and entertainment.

Everyone will have their own AI version of themselves

Meta’s recent release of AI Studio marks a significant leap in the realm of Creator AI. This platform allows anyone to craft an AI avatar, essentially creating a digital twin capable of interacting with audiences on their behalf. For creators constantly grappling with time constraints, these AI characters can enhance engagement by providing personalized interactions that align perfectly with their style and tone.

Key Features of Meta’s AI Studio:

• Customizable Avatars: Creators can design AI agents that mirror their personality and communication style.
• Training on Personal Content: These AI agents are trained using the creator’s existing material, ensuring that the interactions remain authentic and true to the creator’s voice.
• Enhanced Engagement: With AI avatars handling routine interactions, creators can focus more on high-level tasks while maintaining a strong connection with their audience.

This development brings up critical questions regarding content ownership and authenticity. If anyone can create an AI version of a public figure, how do we ensure its legitimacy? For instance, could someone unauthorizedly produce a Bob Cooney AI? This scenario raises pressing concerns:

• Content Ownership: Who holds the rights to an individual’s digital likeness and the content generated by their AI counterpart?
• Authenticity Verification: How can audiences trust that they are interacting with a genuine representation of the creator?

The implications are profound. As AI characters become more prevalent, establishing frameworks for verifying and protecting these digital identities becomes essential. Trust and authenticity will be paramount in this evolving landscape where human-like interactions are increasingly managed by sophisticated algorithms.

The Future of Wearables: Glasses Integrated with AI

Smart glasses are set to change the way we interact with technology, and Meta’s Ray Ban 2 is a perfect example of this trend.

A New Era of Smart Glasses

Zuck is so bullish on this segment, he’s considering having Meta purchase a 5% stake in EssilorLuxottica, the world’s largest eyewear company and maker of Ray-Ban sunglasses. The deal could be worth $5 billion based on EssilorLuxottica’s market value. 

He expects hundreds of different form factors at different price points. “People don’t want to all look the same,” he remarked.

I know quite a few people with the Meta Ray Ban 2, and every one of them seems to really like them. The AI features are just now being added in some countries, and I expect the use cases to grow like the iPhone did in the first few years.

Mark admitted that he didn’t see AI coming before holographic see through lenses.

Did Mark stumble upon the next computing platform? There’s a great story from Nike. They were looking everywhere for the next Michael Jordan to spur their next growth stage. In the meantime they signed a guy who played golf named Tiger Woods. Sometimes the thing you’re looking for comes out of nowhere.

The Future Closer Than It Appears

AI advancements are scary if you’re paying attention. And it’s easy to ignore, as the mainstream tends to focus on issues like the hallucinations of large language models (LLMs) or AI-generated images with seven-fingered hands. Yet, beneath these challenges lies relentless innovation driven by trillion-dollar enterprises.

“The future is already here – it’s just not evenly distributed.” – William Gibson