SmartFace
[Patented US9734809]
Have you ever dreamed of a third hand to control your effects or the knobs on your synthesizers while playing your instrument ? With the SmartFace software and your webcam or the SmartCam camera, turn the most expressive and emotional part of your body — your face — into a true controller. No longer let a simple sine wave modulate your tremolos, phasers, flangers, or filters, and rediscover the joy of full expressiveness by assigning parameters to your mouth, smile, eyes, or eyebrows. Control multiple parameters at once or modulate your effects with your mouth, far more quickly or subtly than you could with your foot, all while keeping your hands on your instrument. Create a unique emotional connection with your audience and offer them a new experience by linking your facial expressions to your sound. Rest assured, the system is precise enough to avoid unwanted facial expressions and only activates when you decide.
SmartCam
The SmartCam is the ideal camera for SmartFace software, designed to provide minimal latency, optimal precision and high reliability. With a frame rate of over 500 frames per second, it guarantees a latency of less than 5 ms, a threshold below which latency becomes imperceptible to professional musicians. Thanks to its near-infrared recording, it ensures precise facial expression recognition and enables reliable use of the SmartFace software, even in concert settings, regardless of lighting conditions. A discreet articulated arm, equipped with a clamp featuring genuine rubber jaws, allows the camera to be positioned and securely fixed to a guitar, bass, or synthesizer stand without the risk of damaging the instrument.
SmartBrain
The SmartBrain is the core of Smart Audio Tools products. It is an electronic module in a pedal format that integrates perfectly on your pedalboard or synth stand. It serves as a platform for Smart Audio Tools' live-performance AI softwares (SmartFace, BeatSync, SmartVinyl and Cloning Machine) and also allows you to use your favorite plugins live!
It includes:
-
an Nvidia CPU and GPU
-
a VST host for your favorite plugins,
-
an 8.9-inch touchscreen,
-
a removable mini 4-switch pedalboard, connected to the SmartBrain via Bluetooth,
-
internal slots for adding electronic cards, allowing it to be upgraded into the SmartWah (see below),
-
low-latency audio inputs/outputs up to 96 kHz:
2 audio inputs
2 audio outputs, -
complete connectivity for controlling your external effects:
1 Expression/CV input
5 Expression/CV outputs
3 switch outputs
1 MIDI input
3 MIDI Out/Thru outputs
4 USB ports (for adding an external sound card or other devices), -
a completely silent cooling system
SmartWah
[Patented US9734809]
The SmartWah is the all-in-one face-controlled effects solution for guitarists, bassists, and keyboardists. It’s a pack that includes the SmartCam and a version of the SmartBrain, equipped with the SmartFace software and an analog Cry Baby-style Wahwah, as well as analog low-pass filters inspired by the legendary Moog and Oberheim (SEM) filters. It also allows you to control all your compatible external effects via its outputs (MIDI, Exp, CV, etc.) and an intuitive touchscreen interface.
Modulate the wahwah with your mouth, much faster and more subtly than with your foot, for striking effects. Create organic synth sounds with a low-pass filter, where the cutoff frequency is controlled by the opening of your mouth, the resonance by your smile and distortion by furrowing your brows. Don’t let a simple sine wave modulate your tremolos, phasers, flangers, or filters: rediscover the pleasure of total expressiveness by assigning parameters to your mouth, smile, eyes, or eyebrows, all while maintaining full control of your instrument.
SmartFace Editor
SmartFace Editor is our facial expression tracking software based on videos, allowing you to transform actor videos into editable facial expression curves for animating 2D or 3D characters. These expression curves can then be exported in various formats for integration into your animation software.
This software was used to create a high-quality database specialized for low-angle shots and near-infrared images, as well as for the training and evaluation of optimized models for the real-time AI algorithm of SmartFace, capable of reaching 500 frames per second.
SmartFramework
SmartFramework is a Python and Qt based library designed to quickly create real-time Python applications utilizing AI, and manipulating audio, video, and various protocols like MIDI, DMX, and OSC. It allows for graphical programming of complete applications by dragging and connecting blocks, similar to Max/MSP, LabVIEW or other graphical environments.
The key difference is that these assemblies are compiled into Python/Qt code, independent of the graphical tool, as if manually written. These compiled objects can be reused in textual code or as new composite blocks. This toolkit was used to create the SmartFace, SmartFace Editor, BeatSync, SmartVinyl, and Cloning Machine software, as well as for our collaborations The Telepathes, Ondoscope, and Zoryas with Claire Williams.
SerializeJSON
SerializeJSON is a Python library for fast, secure, and human-readable serialization and deserialization of Python objects into JSON, designed as a direct alternative to pickle. serializejson is ideal for long-term data management, ensuring that data remains accessible and modifiable through future software updates and evolutions.
-
Supports all objects compatible with pickle.
-
Efficiently serializes binary data using base64 and Zstandard.
-
Provides a human-readable, editable, and versionable JSON.
-
Securely loads data from untrusted sources.
-
universal file format, eliminating the need for rigid file formats.
