Wow, Gestures are the coolest and most hidden feature in MacOS Sonoma, but basically, if you are in any kind of video call (Zoom, Meets, etc.), there is an application-independent way to get gesture recognition.
Enabling it is nearly impossible when you start your video camera, there is a green icon in the menubar and you can now click on it and get a huge number of options.
Note that you can use a Continuity Camera which means your cell phone acts like the camera and it is a much higher quality
Here are the features
- Center Stage. This means that it will follow you as you wander around the room. Not typically useful for staring at a screen. When it is zoomed in, it makes your face kind of round because the lens is a pretty wide angle. Not a good look unless you mount the camera high above your head
- Portrait. This is the blur mode on many systems, but now at the system level.
- Studio Light. This basically lightens your face, I need it on.
- Desk View. This is a special mode, point your phone downward and you can make YouTube-like videos
- Mic Mode. This lets you do
- Reactions. This is the coolest feature and you can use gestures that work across all video meeting products. One thing to note is that certain applications don’t work well with this. For instance, Google Meets with the background effects applied, the gestures won’t get recognized.
Gestures for Apple Silicon or Intel Macs with Continuity Camera
The biggest problem is learning what gestures work and how to make them, here are the. Note the requirements are pretty strict, you need a Mac with Apple Silicon running MacOS Sonoma or later. The document seems to imply that you need to have a Continuity Camera or an internal camera, but I did get this to work with a Logitech USB camera.
If you have an Intel Mac, you need to use Continuity Camera with an iPhone 12, then you can use these gestures. It looks like they are doing recognition on the iPhone (as it won’t work with a generic USB camera):
- Heart. Use your two hands to make a heart sign and hearts will come right out of your hand.
- Balloons. Make a V for Victory sign with two fingers up
- Thumbs Up, this causes an emoji to pop up
- Thumbs Down produces that icon
- Rain. Two Thumbs down with both hands and your background rains
- Confetti. Both hands show the victory sign with confetti background
- Fireworks. This is thumbs up with both hands
- Lasers. If you make the horn sign (the forefinger and pinky up) and you get Lasers in the background
Don’t use a background in your Zoom, Meets, or Teams application
This is noted somewhere but using a background doesn’t work as well, it doesn’t seem to deal with the gesture icons and things, they appear but are not clear.
Presenter Overlay
This just requires an Apple Silicon Mac and basically, it detects when you are screen sharing with a video application:
- Turn on the screen-sharing feature, then the magic icon will appear in the menu bar
- If you click on Large, use the video preview and you can see the effect where you appear right there.
- There is a small version as well where your presence is small.
Reactions on iPhone FaceTime only
You can get this only if you use FaceTime Reactions on an iPhone. This is not a general facility like it is on the Mac (but of course you have an iPhone).
