To get the best use of Background Removal(what we also call User Segmentation or Dynamic Greenscreening) there are two main variables to always take into account: Lighting and Background. Showing is always better than telling, so I’ll speak to tips/tricks on optimizing those two variables through a few video examples, along with a few other related tips.
Tips for Your IRL background
You’ll notice in the first video above that I am wearing a black hoodie. This contrasts nicely with some of my background, but there is a black door behind me which is the exact same color as the clothing. When I first select “Hide Background” to initiate background removal, you can see that I have a nice, clean cutout of my persona - even in the area where my persona/sweatshirt closely matches the background. But when I start moving, this introduces some artifacting at that point.
The solution is simple:
As you can see above, I simply rotated my laptop/camera 45 degrees, to maximize contrast between myself and my IRL background. In this case, even a little shoulder shimmy, or even full-fledged dancing, works great!
So the top takeaway is to point your camera such that you get as much contrast as possible between you and your actual background. I have a few different shirts here at the office, and depending on if I’m going to do a quick demo in the coffee shop downstairs, or in our conference room, or in my office lobby, my desk, or wherever, I’ll always be prepared with clothing that helps me achieve the best quality regardless of environment.
Tips on Lighting
Beyond contrast with your background, the other top tips are around lighting. As with any video technology, you can’t use User Segmentation technology in the dark. The beauty of Background removal is that you don’t need super expensive, studio-quality lighting to have a top notch experience. What you do need is some uniform overhead lighting, like in the this video:
With some solid background contrast and decent overhead lighting, your persona will look crisp and fly.
The video above also shows two other points I wanted to make about using background removal/replacement. Major illumination and exposure changes will throw us off, momentarily. When I quickly move my camera from directly onto the light source, and pan down to me, the camera hasn’t quite adjusted to the lighting difference. So it takes a moment for our AI to get its bearings and remove your background with a well-lit persona.
Secondly, if the camera ever loses you, like if you step away from the PC for a snack break and then sit back down or have major illumination changes, make sure that your face is visible. That’s where ChromaCam starts, and then be patient for a second as ChromaCam’s AI learns and adjusts to you at first.
The other big lighting tip is what you DON’T want to do, which is to locate your lighting source directly behind you, as in this clip:
When my persona is covering the ceiling lamp directly behind me, I'm extracted from my background just fine. But when I move, this causes the backlighting to be in the immediate vicinity of my persona, which causes issues. And the issues are especially pronounced when the backlighting is around the head/hair.
To close, there are a few last things to note. Dynamic background removal is optimized for “head and shoulders”, and so if you move your hands around the frame you might have issues with your hands disappearing. Though this won’t really matter much, since typically your hands will be occupied with a controller/keyboard/input device for the game you’re streaming.
You can also wring a little bit extra computing efficiency by adjusting the resolution settings of your stream. A 720p stream is still being pulled even when your persona is small, so adjusting the resolution settings will ensure you’re using less compute to process a small persona.
If you can learn to work with those two variables, you will take care of 99.9% of any quality issues associated with virtual greenscreening with our technology. Getting the lighting and the background right really is the Nuts and Bolts of making high quality content with Personify.
Have a great time with Dynamic Greenscreening!