Spinor Shark 3D Live

Virtual Production Solution

Shark 3D Live Virtual Production Solution provides you a comprehensive solution for live virtual production based on interactive virtual simulated worlds (as known from video games). The director gets results live and gains a wider scope for creativity, experimenting, testing and tweaking:

• Live on-set visualization and live previz
• Live on-set virtual production and live VFX

Designed for agility and flexibility from the bottom up, Shark 3D Live uniquely combines the possibilities of movie, game, broadcasting and virtual reality production.

Create and run interactive virtual 3D worlds

• Joining forces: Movie, game, broadcasting and virtual reality production - Shark 3D Live combines possibilities and features of professional movie production software, high-end game technologies and broadcasting production solutions. This unique combination opens new levels of creativity and productivity in each of these areas.
• Flexibility and agility without dead ends - Shark 3D Live is designed for agility and flexibility from the bottom up. It includes a real-time 3D authoring editor and a cross-platform 3D engine. It can also be used as middleware and software platform.
• Proven reliability - Shark 3D Live is used successfully in a large variety of projects on various software and hardware platforms in areas like video games, films and series, broadcasting and interactive applications.

Authoring editor

• Tree-based scene setup
• Hierarchical organization of nodes
• Lifabs: Smart live prefabs and templates for reusing entities and functionality
• Unlimited nesting of Lifabs
• Live editing of Lifabs
• Customization of Lifabs via slots
• Support for defining custom level managements
• Support for defining new 3D states and custom view ports
• Inspection of run-time instances of Lifabs
• Python shell for controlling the editor
• Python shell for interactively manipulating runtime components
• Python scripts can control the editor in window-less command line mode

Recording, track editing, and rendering

• Feature-based recording and replaying actions in the 3D world
• Per channel control over what is recorded, what is replayed, and what is calculated or simulated live at the same time
• Recording and replaying sub-channels, for example replaying a raw camera movement combined with real-time generation of steadicam movements with different settings
• Recording and replaying resulting channels, for example total camera movements including steadycam movements
• Physics-based replaying, which can react to and influence other entities or the environment
• Adaption of run-time simulation to edited recordings, for example adapt walk animations live to an edited character path
• Slow motion and time lapse simulation, recording and replaying
• Hierarchical structure of tracks and shots
• Cutting, blending, timing, and scaling of tracks
• Non-destructive track editing
• Non-linear track editing
• Track editing on any hierarchy level like the whole scene, groups of characters and animated objects, individual characters and animated objects, groups of animation channels, or individual animation channels
• Block-based interleaving sparse track storage optimized for real-time reading, writing, seeking and live non-destructive recombining high numbers of channels containing high numbers of key frames
• Large recordings don't need to fit into memory but are automatically streamed to and from disk when recording, playing or editing

Output

• Packaging as stand-alone application (e.g. game)
• Live low latency H.264 (raw or fragmented mp4) streaming via TCP and WebSocket
• Render to a sequence of high resolution image files
• Render to high resolution video file, supporting different codecs
• Semi-real-time GPU-based rendering with enhanced quality settings
• Virtual reality head-mounted display headsets, including six degrees of freedom rotational and positional tracking
• 360 degree panorama rendering
• Stereo 3D rendering
• Multi-channel rendering

Input

• Gamepads, joysticks, and other controllers
• Video-in
• Camera tracking
• Camera lens distortion correction and calibration
• Camera lens focal length measurment
• Control input via network
• Synchronization with video-in, genlock and tracking, including individually configurable delays with buffering
• APIs for integrating and synchronizing any input source

Cloud support

• Server-side processing (including rendering), streaming input from and output to thin clients
• Multiple isolated or collaborative users per server-side engine instance
• Extensive resource sharing between multiple users, reducing clould infrastructure costs to a mere fraction

Renderer

• All features are completely real-time (no precalculation waiting times after scene changes)
• All kinds of objects are handled in the same way (no need for differentiating between static and dynamic objects)
• Physically based rendering including energy conservation, albedo, reflectivity, glossiness and Fresnel
• Reflectivity workflow, metalness workflow and custom workflows
• Light-transport-based rendering of soft lighting
• Omnidirectional and spot lights
• Extended light sources casting contact hardening soft shadows
• Reflection and refraction of flat and curved surfaces
• Global illumination effects like interreflection, color bleeding, and ambient occlusion
• Lighting of complex materials like strands of hair, skin, fabric or brushed metal
• Efficient detailed shapes like fur or hair
• High dynamic range effects
• Bokeh depth of field, with efficient support of large apertures, and without halo or bleeding
• Bloom
• Particles
• Fog
• Node-based shader editor
• Physically based rendering can be accessed via multiple nodes which can be used and combined flexibly in different ways for special needs
• TrueType Unicode text rendering
• Extruded 3D text, based on TrueType Unicode, including bevels and mapping coordinates
• Shader animation
• Chroma keyer
• Video textures (video plates)
• All per-vertex and per-pixel rendering operations are running completely on the GPU

Animation system

• Node-based animation system
• Hierarchical animation fading and blending
• Import of path animations
• Path animation editor integrated in authoring editor
• Import of skeleton animations
• Flexible skinning of multiple objects by multiple skeletons
• Any number of attachment points of skeletons
• Retargeting animations to different skeletons
• Walk animation adaption in all directions
• Mesh animations (morphing)
• Compact storage and efficient execution of sparsely and intermittently animated meshes
• Blend shapes (morph targets)
• Unlimited number of simultaneous active blend shapes
• Compact storage and efficient execution of overlaying many sparse blend shapes
• All per-vertex and per-pixel animation operations are running completely on the GPU

Audio

• 3D sound
• Physics collision and friction sound effects

Physics

• Collision geometry built of collision meshes and primitive geometries
• Various types of constraints like for example ball-and-socket, hinge and hinge-two, or for keeping a character vertical
• Motors and character movers
• Player control elements
• Physics driven skinning
• Ragdoll
• Networking integrated with physics

Networkign

• Integrated networking
• Server-side versus client-side execution
• Lag suppression for player movements
• Remote command execution
• Optimizations for avoiding network traffic when loading levels
• Adaptive network traffic compression

Logic

• Command handling and dispatching
• Generic triggers and sensors
• Sandbox based scripting language "Perch" designed for easily accessing engine components and integrated protection against memory leaks and crashes
• Python

Component architecture

• Component architecture based on composition, encapsulation, name spaces, dynamic command flows, and scripting bindings
• Thread-safe architecture from bottom up
• Thread-safe dependency graphs supporting delayed evaluation
• Support for streaming, background loading, and world part states onto disk in separate thread
• Generic dynamic method invocation APIs for C++ components, which can be accessed for example via Python
• Universal and flexible serialization mechanism
• Support for adding serialization for existing external C++ objects which are not prepared for serialization
• Resource manager abstraction supporting individual files, pack files, and TCP
• Universal efficient hierarchical snake data file format, with equivalent text and binary representations, optimized for efficiency and simplicity
• Unicode based (UTF-8) from the bottom up
• SAP (Simple Access Protocol) as generic and easy-to-parse remote method invocation protocol, supporting also callback objects

Target platforms

• PC (Windows, Linux)
• Mobile devices
• Consoles
• Browser plugin

Documentation

• Introduction articles
• Tutorials
• Reference sections
• Keyword search
• Context sensitive integration with the authoring editor
• As HTML (~25k pages) or pdf (~10k pages)

Reliability

• Based on very high software design and coding standards
• Focus on clean and robust software architectures
• Designed from the bottom up for stablility without compromising performance

Software platform

• Designed, from the bottom up, as an universal software platform for creating new modules, features, customizations, extensions and live connections to other software
• Designed for flexibility from the bottom up
• Modular system for a large variety of applications in different markets
• High-level Python APIs
• Low-level C++ APIs
• Custom modules can access and use the same APIs as all our standard Shark 3D component implementations (we at Spinor don't use hidden internal APIs)
• Custom modules can access standard Shark 3D components and vice versa
• Live-Live Editing (LLE) to update changes in the Python editor code (e.g. custom editor nodes) live into the running editor for fast customization testing turn-around times of the editor
• Generic and flexible editor development toolkit for creating own customized editors
• APIs for defining new recording and replay channels
• Flexibility of defining how replaying recordings interact with the simulation of the 3D world
• Generic synchronization architecture
• Custom rendering components can access the whole scene database and can interact with all rendering steps
• Custom animation components can access, modify or build new animation, skeleton and bone transformation dependency graphs
• APIs for live input of animation data from other software, for example live motion capture data
• Custom components can implement individual custom networking synchronization algorithms and protocols
• Generic APIs for custom script language bindings
• Generic APIs to asset and project management software
• TCP protocol for live updating external run-time resources

Software achitecture

• Clean modular multi-layer achitecture
• Parallelized Dependency Graph (PDG) processing the virtual 3D world live efficiently
• Rapid Construction Kit (RCK) manipulating and controlling the Parallelized Dependency Graph
• Live incremental translation of nodes and Lifabs into Rapid Construction Kit components
• Re-entrant Modular Shader System (RMSS)

Renderer hardware

• Rack mountable in 4U chassis
• Processor: Intel Core i7 4790K 1150, 8 MB Cache, 4,0 GHz
• Main memory: 32 GB DDR3 memory
• Graphics card:
  GeForce
  PCIe ASUS STRIX-GTX 980 TI
  GM200-310-A1 "Maxwell"
  Processor rate 1216MHz
  Memory 6GB GDDR5, 1800MHz
  384bit, 346GB/s
  Shader units /TMUs/ROPs: 2816/176/96
  Computing power 6849 GFLOPS (Single), 214 GFLOPS (Double)
• Video in/out:
  DeckLink 4K Extreme 12G
  Capture and playback 12G-SDI in SD, HD, Ultra HD and 4K DCI up to 4096 x 2160 60p in 10-bit YUV or 12-bit RGB
• Storage: 500 GB SSD and 4 TB HDD

Video plates

Multiple video plates up to 4096x4096, GPU-accelerated decoding