3d models in retail at pervasive scale 3d commerce: the
TRANSCRIPT
© The Khronos® Group Inc. 2021 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License
3D Commerce: The art and business of using 3D models in retail at pervasive scale
Nathaniel Hunter - Dreamview Studios, The Khronos GroupMax Limper – DGG, The Khronos GroupBrian Freisinger - Dreamview Studios, The Khronos GroupJatinder Kukreja - SuperDNA 3D Lab, The Khronos GroupPaweł Nikiel – CGTrader, The Khronos Group
© The Khronos® Group Inc. 2021 - Page 2This work is licensed under a Creative Commons Attribution 4.0 International License
Today’s Speakers
Nathaniel HunterDreamview
Jatinder KukrejaSuperDNA 3D Lab
Paweł NikielCGTrader
Brian FreisingerDreamview
Max LimperDGG
© The Khronos® Group Inc. 2021 - Page 3This work is licensed under a Creative Commons Attribution 4.0 International License
Nathaniel Hunter• Co-Chair of 3D Commerce Asset Creation TSG
• 30 years in content creation spanning film, TV, commercials, games, products, and internet technology
• Chief Executive Officer at DreamView, Inc.
- B2B online platform serving retailers, suppliers, manufacturers and brand owners
- Scalable and standardized management, creation and distribution of real-time and photo-real product content
pawelnikiel.com@pawel_nikiel
cgtrader.com
© The Khronos® Group Inc. 2021 - Page 4This work is licensed under a Creative Commons Attribution 4.0 International License
Max Limper• Co-Chair of 3D Commerce Asset Creation TSG
• glTF spec contributor (binary glTF, 2.0 PBR model)
• PhD in CS on 3D data optimization(Fraunhofer / TU Darmstadt)
• CTO & Co-Founder at DGG, creating RapidCompact- Commercial 3D asset optimization solution
serving leading retailers & 3D studios
- Tech & workflows aligned with standards(glTF, USDZ, Asset Creation Guidelines)
- Additional open-source scripts on GitHub max-limper.de@mlimper_cg
rapidcompact.com
© The Khronos® Group Inc. 2021 - Page 5This work is licensed under a Creative Commons Attribution 4.0 International License
Brian Freisinger
• 30 years in content creation spanning film, TV, commercials, games, products, and internet technology
• CG Supervisor at DreamView, Inc.
- B2B online platform serving retailers, suppliers, manufacturers and brand owners
- Scalable and standardized management, creation and distribution of real-time and photo-real product content
© The Khronos® Group Inc. 2021 - Page 6This work is licensed under a Creative Commons Attribution 4.0 International License
Jatinder Kukreja
© The Khronos® Group Inc. 2021 - Page 7This work is licensed under a Creative Commons Attribution 4.0 International License
Paweł Nikiel• Contributor to 3D Commerce WG
• Over 10 years of experience in 3D art,VR & AR development, pipeline & automation,data visualization
• Everything interactive (WebGL, game engines etc.)
• 3D Automation Lead at CGTrader
- Batch model processing and validation
- Tools development
- Automated assets generation
- Interactive assets developmentpawelnikiel.com
@pawel_nikielcgtrader.com
© The Khronos® Group Inc. 2021 - Page 8This work is licensed under a Creative Commons Attribution 4.0 International License
Impact of 3D in Retail
“...46 percent of retailers planned to deploy either AR or VR solutions to meet customer service experience requirements….”
“...the impact of AR or VR in retail can be transformative...”
“...retailers can use AR as an extension of the brand experience to engage customers in immersive environments and drive revenue…Additionally, AR can be used outside the store
after a sale to increase customer satisfaction and improve loyalty...”
April 2019 Gartner press release
© The Khronos® Group Inc. 2021 - Page 9This work is licensed under a Creative Commons Attribution 4.0 International License
Let’s Think About “Search” as a Use CaseE.g. Google Search now enables 3D and AR experience directly from Search results
View in 3DRetailer’s Content
View in AR
Retailer’s Website
Buy Now
Link Retailer’s Content Link
© The Khronos® Group Inc. 2021 - Page 10This work is licensed under a Creative Commons Attribution 4.0 International License
View in 3D
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
…. Across different platformsInconsistent visual quality across experiences
© The Khronos® Group Inc. 2021 - Page 11This work is licensed under a Creative Commons Attribution 4.0 International License
Where Does 3D Content Come From ?
View in 3D
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
Retailer’s Content
View in AR
Retailer’s Website
Buy NowRetailer’s Content
3D Content Production
© The Khronos® Group Inc. 2021 - Page 12This work is licensed under a Creative Commons Attribution 4.0 International License
3D Content ProductionOne example of the content production process
Manufacturer RetailerEnd-User
Platform
Content Creator
3D Tools
Uses
Uses
Distributes via Publishes on
Uses View in 3D
Search
© The Khronos® Group Inc. 2021 - Page 13This work is licensed under a Creative Commons Attribution 4.0 International License
3D Commerce - The Promise
IKEA catalog uses augmented reality to give a virtual preview of furniture in a room – August 2013!
Early Experience ShowsIncreased customer engagement!
Strengthened brand loyalty!Deeper product understanding!
More online sales!Fewer returns!
So why is 3D Commerce taking so long to become pervasive?
=$$$!
© The Khronos® Group Inc. 2021 - Page 14This work is licensed under a Creative Commons Attribution 4.0 International License
3D Commerce Initiative – the Opportunity
View in 3D
table lamp
table lamp
Enable 3D Commerce to achieve deployment at industrial scale Industry cooperation between technology and retail leaders
Interoperability standards, guidelines and Certification Programs
© The Khronos® Group Inc. 2021 - Page 15This work is licensed under a Creative Commons Attribution 4.0 International License
State of the Industry Before 3D Commerce
Products don’t come with 3D data – and I can’t physically scan
them all fast enough!
The green couch looks blue on some devices – lots of product returns
are expensive!
CAD tools don’t let me easily generate
the data I need for E Commerce!
Many 3D products on my ecommerce
web-site first appear upside down! I have to
hand-tune 1000s of models!
I wish I had high quality, realistic 3D models for virtual
promotional photoshoots!
Everyone defines their product data for sizes and colors differently
– nothing is consistent!
I need the materials in my 3D models to look
completely realistic!
Complex retail pipeline with hundreds of companies and thousands of productsMany friction points - tooling, technical and commercial
© The Khronos® Group Inc. 2021 - Page 16This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Mission
Create specifications and guidelines to align the 3D asset workflow from product design through manufacturing, through
each stage of retail to end-user delivery platforms
© The Khronos® Group Inc. 2021 - Page 17This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Mission
© The Khronos® Group Inc. 2021 - Page 18This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Active Initiatives
3D GraphicsDesktop, Mobile
and Web
3D Assets Authoring
and Delivery
Portable XRAugmented and Virtual Reality
Parallel ComputationVision, Inferencing,Machine Learning
Safety Critical APIs
© The Khronos® Group Inc. 2021 - Page 19This work is licensed under a Creative Commons Attribution 4.0 International License
Diverse Technology and E-commerce Companies Working Together
© The Khronos® Group Inc. 2021 - Page 20This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Goals
Visual realism and consistency no matter where
the model is displayed
© The Khronos® Group Inc. 2021 - Page 21This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Goals
Product display configurability with
consistency and authenticity
© The Khronos® Group Inc. 2021 - Page 22This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Goals
Structured metadata for product management and configurability of viewing
© The Khronos® Group Inc. 2021 - Page 23This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Goals
Reduce production, distribution and marketing costs
Guidelines for tools and product designers to create assets with consistent data to be used through the 3D Commerce pipeline
© The Khronos® Group Inc. 2021 - Page 24This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos 3D Commerce Goals
Reduce production, distribution and marketing costs
Visual realism and consistency no matter where
the model is displayed
Product display configurability with
consistency and authenticity
Structured metadata for product management and configurability of viewing
Guidelines for tools and product designers to create assets with consistent data to be used through the 3D Commerce pipeline
© The Khronos® Group Inc. 2021 - Page 25This work is licensed under a Creative Commons Attribution 4.0 International License
Images: OTTO, IKEA, Shopify
IKEA Place
OTTO yourhome
AR in Shopify
AR in E-Commerce Today!
Which of these is the “real one”? Source: Shopifyhttps://www.youtube.com/watch?v=ajF5Rasyq3o&t=428s
• Technology is there!
© The Khronos® Group Inc. 2021 - Page 26This work is licensed under a Creative Commons Attribution 4.0 International License
?3D assets for real-time apps:AR + Web technology
Images: IKEA
Offline CGI
Large Files
Complex Materials
Millions of 3D AssetsReal-T
ime Rendering
Compact Files
Real-Time PBR Materia
ls
The “Offline / Real-Time Content Gap”
© The Khronos® Group Inc. 2021 - Page 27This work is licensed under a Creative Commons Attribution 4.0 International License
Optimization “recipes” for various sources
3D-Scanned Assets• Simplification
• Material tweaking & texture baking
CAD data• Tessellation & geometry cleanup
• Rule-based, automated materials assignment
Assets from 3D modeling (DCC tools)• Simplification
• Material conversion & texture baking
[2] https://rapidcompact.com/case-study-markgraph/[1] https://twitter.com/nebulousflynn/status/1372589608165384201
[3] https://rapidcompact.com/case-study-otto/
1
2
3
© The Khronos® Group Inc. 2021 - Page 28This work is licensed under a Creative Commons Attribution 4.0 International License
Scalable 3D Today: Two Secret Ingredients
Automate 3D optimization tasks
• Use “recipes” for different archetypes of 3D assets / sources
• Automate creation of variants for publishing targets,without the artist having to care about it (“YouTube approach”)
• Enable easy upstream modifications (easy to re-run where necessary)
Use Web standards, such as glTF
• High compatibility
• Support across different DCC toolchains & viewers
• Large ecosystem (including many free tools!) available
© The Khronos® Group Inc. 2021 - Page 29This work is licensed under a Creative Commons Attribution 4.0 International License
3D Commerce Deliverables
Extensions
PBR MaterialsTransmission
ClearcoatSheen
RefractionSpecular Color
Color AttenuationVolumetric Properties
Consistent Asset Creation
3D Commerce Viewer Certification
Program
Free guidelines for tools and product designers to create high-quality
real-time 3D models for e-commerce
Material VariantsEfficient
transmission of many e-commerce
products
Open-Source 3D Viewers
Retailer 3D Viewers
Non-retailer 3D Viewers
Non-Retailer Web use cases
Social, Ads, Search …
Retailer WebStore Fronts
Brands /Manufacturers
Future ExperiencesWeb, Apps,
Mobile, AR ...
Viewers certified to accurately display 3D productsConsistent deployment in a wide variety of end-user
experiences: Web, social media, ad platforms, apps etc.
End User Experiences On the Web and in Apps
Desktops and mobile RenderingTechnologies
How to use KTX 2.0 supercompressed textures
for minimized glTF download size at high quality
MetadataSupports for XMP Extensible
Metadata Platform ISO 16684-1 to hold consistent data for
product discovery, dimensions, how a virtual product interacts
with the physical world, defining variants, etc.
Certification to go beyond testing rendered pixels and
achieve ecosystem consistency around tone mapping and
display calibration
Open-source asset checker tool in design to guide creation and assess compliance of reliable
e-commerce 3D assets
glTF Roadmap discussions underway: more PBR
materials, realistic cloth and apparel, constraints for
simulation etc… Metadata conventions
Next Steps
© The Khronos® Group Inc. 2021 - Page 30This work is licensed under a Creative Commons Attribution 4.0 International License
Takeaways
▪ There are a lot of 3D scans, CGI & CAD data sets to be made “real-time ready”
▪ Automating optimization is key!
▪ Relying on 3D standards is key!
© The Khronos® Group Inc. 2021 - Page 31This work is licensed under a Creative Commons Attribution 4.0 International License
Example Product
© The Khronos® Group Inc. 2021 - Page 32This work is licensed under a Creative Commons Attribution 4.0 International License
Asset Creation, QC & QA
© The Khronos® Group Inc. 2021 - Page 33This work is licensed under a Creative Commons Attribution 4.0 International License
Color Variations
© The Khronos® Group Inc. 2021 - Page 34This work is licensed under a Creative Commons Attribution 4.0 International License
Photo-real use cases
© The Khronos® Group Inc. 2021 - Page 35This work is licensed under a Creative Commons Attribution 4.0 International License
Example Product
© The Khronos® Group Inc. 2021 - Page 36This work is licensed under a Creative Commons Attribution 4.0 International License
Asset Creation, QC & QA
© The Khronos® Group Inc. 2021 - Page 37This work is licensed under a Creative Commons Attribution 4.0 International License
Variations
© The Khronos® Group Inc. 2021 - Page 38This work is licensed under a Creative Commons Attribution 4.0 International License
An Example Process
© The Khronos® Group Inc. 2021 - Page 39This work is licensed under a Creative Commons Attribution 4.0 International License
DEMONSTRATION
Showcasing real world problems and solutions with 3D content for:- 3d Commerce Guidelines based modeling and texturing- Photo-real products- Augmented Reality
© The Khronos® Group Inc. 2021 - Page 40This work is licensed under a Creative Commons Attribution 4.0 International License
Benefits of automated processing
DGG RapidCompact Customer Case: Preparing 5,300 CGI assets for the Web/AR (glTF & USDZ)(3D models of furniture products)
Estimated manual effort: 2,650+ Hours(30 mins to 1h manual work per asset)
Automated optimization: 54.7 Hours ● Polygon reduction● Texture baking (incl. AO)● Scene graph flattening● Draw call reduction● Compression (Draco) & glTF export
© The Khronos® Group Inc. 2021 - Page 41This work is licensed under a Creative Commons Attribution 4.0 International License
Automating optimization for real-time applications
Original Image: Khronos Asset Creation Guidelines v1.0https://github.com/KhronosGroup/3DC-Asset-Creation/blob/main/asset-creation-guidelines/full-version/sec99_PublishingTargets/PublishingTargets.md
© The Khronos® Group Inc. 2021 - Page 42This work is licensed under a Creative Commons Attribution 4.0 International License
Case study - DreamView & DGG
Read the full story here:https://rapidcompact.com/siggraph2021-demo-production-to-real-time/
CGI Asset
Real-Time Asset
Try it outyourself!
© The Khronos® Group Inc. 2021 - Page 43This work is licensed under a Creative Commons Attribution 4.0 International License
Case study - DreamView & DGG
Read the full story here:https://rapidcompact.com/siggraph2021-demo-production-to-real-time/
From V-Ray to Physical Material
Polygons: 500K to 20KDraw Calls: 63 to 6File Size: 180MB (FBX) to 9MB (glTF)Processing Time: 38s
Real-Time Asset(“High Resolution” Version)9MB Optimized glTF
© The Khronos® Group Inc. 2021 - Page 44This work is licensed under a Creative Commons Attribution 4.0 International License
Jack Asteroid in AR!
Read the full story here:https://rapidcompact.com/siggraph2021-demo-production-to-real-time/
Converting Jack Asteroid into an AR versionAnimation baking and FBX export, compression with RapidCompact to glTF & USDZ
© The Khronos® Group Inc. 2021 - Page 45This work is licensed under a Creative Commons Attribution 4.0 International License
Jack Asteroid in AR!
Read the full story here:https://rapidcompact.com/siggraph2021-demo-production-to-real-time/
Try it out yourself!
© The Khronos® Group Inc. 2021 - Page 46This work is licensed under a Creative Commons Attribution 4.0 International License
Challenges & Opportunities
Jatinder @ SuperDNA 3D Lab
© The Khronos® Group Inc. 2021 - Page 47This work is licensed under a Creative Commons Attribution 4.0 International License
The future of 3D Commerce
▪ Enable users to configure their own products.
▪ Enable users to design their own products.
▪ Let the user play with your product - make it interactive!
© The Khronos® Group Inc. 2021 - Page 48This work is licensed under a Creative Commons Attribution 4.0 International License
Get Involved in Khronos 3D Commerce!3D Commerce BOF Tomorrow with lots more details
3D Commerce: The Art and Business of Using 3D Models in Retail at Pervasive ScaleTuesday, August 10 from 3-4:30pm PDT
Join Khronos and the 3D Commerce Working GroupWe welcome both commerce and technology companies
Influence and gain early access to extensions, tools, programs
Or join the 3D Commerce Advisory ForumProvide requirements and feedback to the working group
Free of charge in any interested companyhttps://www.khronos.org/3dcommerce/
Tuesday, August 10 from 3-4:30pm PDT
© The Khronos® Group Inc. 2021 - Page 49This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Group BOF Series at SIGGRAPH
• OpenXR: On an HMD Near You Now!- Tuesday, August 10 from 1-2pm PDT
• 3D Commerce: The Art and Business of Using 3D Models in Retail at Pervasive Scale- Tuesday, August 10 from 3-4:30pm PDT
• An Exploration of Pervasive 3D in Geospatial with glTF- Wednesday, August 11 from 10-11am PDT
• Vulkan Without Boundaries: Case Studies from the Real World- Wednesday, August 11 from 1-3pm PDT
• Hands on with glTF: Making 3D pervasive- Thursday, August 12 from 10-11:30am PDT
www.khronos.org/events/2021-siggraph
© The Khronos® Group Inc. 2021 - Page 50This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 50This work is licensed under a Creative Commons Attribution 4.0 International License