How to Replace a Screen in After Effects CC

In this tutorial, you’ll learn how to replace a green screen on a device in After Effects CC. We’ll cover two common scenarios: how to replace a static screen and a moving screen.

When does this skill come in handy? Basically, if you’re filming somebody using a screen like a tablet or a smartphone, you may not always be able to shoot exactly what’s on the screen in real-time. Maybe you don’t have the visuals ready before filming, or coordinating the movement of everything is too complicated. Or maybe you just can’t get it to look good filming on a screen on location. Whatever the case, you’ll need to place your visuals onto a screen in post-production. Let’s dive into how to make this happen!

Part 1: How to Replace a Static Screen

Step 1: Create a New Composition

Import the footage you want to work with. You can grab something from Motion Array’s stock video library. Find a shot of a person with their tablet on a table, swiping through some imaginary images.

1. Trim your footage to only what you’re going to be using. You’re going to be adding some effects that will make After Effects work pretty hard. The fewer unnecessary frames you have to render, the better. 
2. Create a new composition by going to Composition > New > Composition. This will act as the screen of the tablet so set the dimensions to be an approximate size of a screen. You might need to unlock the aspect ratio to make it work. 
3. Name it so that you’ll remember it’s the screen composition, and hit OK.

Step 2: Create a Solid to Match the Screen

The next step is to create a Solid that will double as the screen so you can line everything up nicely. Later on, you will replace this Solid with whatever you want to be on the screen, so the more precise you can be with this design the better the final result.

1. Add a Solid that will contrast with the green easily, such as red. Go back to your main composition, and drag your screen composition into your main project.
2. Go to Effects > Corner Pin, and drag it onto the composition layer. When it’s highlighted, you should be able to see dots on the corners of your layer. If not, make sure layer controls are checked by going to View > Show Layer Controls.
3. Drag corners to match up the screen. It’ll help to drop the Opacity of your screen composition so you can see both at the same time. Scale it up slightly so it’s a bit larger than the actual screen. A little too big is better than too small.
4. Bring the Opacity back up.
5. Now go into your screen composition. Drag your chosen image above your red layer and resize it. Whatever you see here is what you’ll see on the screen.
6. Hide the red layer instead of deleting it in case you need to re-adjust things later. You can also add video if you’d like, and it’ll work the same.

Step 3: Rotoscope Objects that Move in Front of the Screen

When you go forward here you will see a problem: the subject’s finger goes beneath the picture on the screen.

There are two main methods to fix this. One is to keep the photo on top and rotoscope the finger to out of each frame individually. This takes a lot of time, so take advantage of the fact that the tablet has a green screen.

1. Drop the screen comp below your footage and add the Keylight Effect to the footage layer with the tablet.
2. Select the eyedropper tool and click on the green of the screen.

In just one click you can see that it does not too bad of a job. But there are a couple of problems you’ll need to address.

First, the green didn’t entirely key out as there’s some strange glare and banding left over. There’s also some blur on the finger as it moves through the image. To resolve these: 

1. Go to View in the Keylight Effects panel, and set it to Screen Matte. Now you should see a black and white representation. Black is transparent and white is opaque. You want to focus on the screen for now and make the screen as black as possible.
2. Go to the Screen Matte options and move the Clip Black slider until it’s completely black with no banding.
3. Reduce the Clip White slider until it becomes a lot better outside the screen. In this case, you don’t have to worry about the outside too much because you’re going to be isolating the screen in a second. But in other cases, it’s good to get everything as black and white as possible.
4. Go to View > Final Result in the Keylight Effects and you’ll see it’s a lot better. But there may still be some discoloration when it gets blurry moving from right to left. 
5. Go to View > Status in the Keylight Effects panel. Play around with the Clip Black and Clip White sliders until you reach a happy medium and the finger has no grey edge.

Step 4: Isolate the Green Screen Only

In taking away the green from the screen, you might also take it from some other green elements in the footage. So, how do you isolate this effect to the screen only?  

1. Select the actual Footage Layer and hit Q to bring up the Rectangle tool.  
2. Click and drag over the screen to isolate it. 
3. Now go down to your Mask in your layer and set it to “none” to deactivate it.
4. Tell only Keylight to use it by going to Inside Mask, choosing Mask 1, and then hitting invert. Now the green outside of the mask is returned and you will still key out the image.

Step 5: Animate the Screen Picture

So now the effect should be looking pretty good. But why don’t you take it a step further? Animate the picture so it looks like the person is swiping the image off-screen:

1. Go to the first point of contact that the finger makes on the screen. 
2. Head to your Screen Comp. Set a keyframe for this position then move the playhead forward to the place where you want your image to be fully off-screen. 
3. Dive back into your Screen Comp and keyframe your image offscreen.
4. Highlight the keyframes and add an Easy Ease. You can finesse these further by highlighting your keyframes and opening the Graph editor. From here, you can tweak your motion to perfection.
5. Add the finishing touch by selecting the Motion Blur option and enabling it for that image layer specifically. 

Step 6: Refine the Effect

If you place in another image for realism, like that person is scrolling through images, it looks pretty good. But if you study this image in-depth, you can tell it’s a little too perfect to be projected from a screen being recorded. 

In a pinch, this will be good enough, but there are two ways to improve the effect further.

Add a Camera Lens Blur Effect

1. First, add a Camera Lens Blur effect. 
2. Blur your footage out subtly, just 1-2 pixels of blur, so it doesn’t look like the screen is in perfect focus all the time. Easy.

Add Screen Glare

The second method might not be as intuitive, but it’s a cool result. Try adding some light across the screen, from an off-camera source.

1. Go to your Screen Comp and add a white solid on top.
2. Create a Mask for the solid on part of the screen, shaped like a Screen Glare.
3. Add a subtle Feather to it and drop the opacity to below 25%. 

It’s subtle but it makes the whole thing feel a little bit more real. 

And that’s how you replace a screen that isn’t moving. But what if it is? For instance, if the camera is moving, or if someone’s holding a phone screen that isn’t rock steady?

Part 2: How to Replace a Moving Screen

To actually make any sort of screen replacement work, you’ll need to track the movement of the screen exactly. Now, if you can see all the corners of your screen, a traditional perspective corner pin track could work. You can learn more about that here.

But in a shot where someone is interacting with the blue screen, the corner pin can be very easily disturbed. So instead, use an incredibly powerful tracker that comes with After Effects: Mocha.

Step 1: Prepare the Footage in mocha AE

1. With your footage on your timeline, go to Animation > Track in mocha AE.
2. Mocha should recognize the footage and automatically set up your composition for you.  If it doesn’t, export your footage in H.264. That’s one that Mocha will read easily. Then bring it back onto your timeline and keep going with our next steps.

Step 2: Outline the Shape of the Screen

1. In the new mocha timeline, bring your footage playhead back to the start. 
2. Select the x-spline tool.
3. Click and make a basic shape around the screen of your phone. As long as the phone is within the confines of your mask, it’s pretty good at doing the rest of the work for you. You’re just telling Mocha where to look and the rough shape of your object. Once you’ve finished your last dot, hit the right mouse button to complete your outline.

Step 3: Track the Screen in mocha AE

1. Select Show Planar Surface. Now a box comes up and you’re going to use this to tell mocha exactly where the screen is on the phone.
2. Just click and drag the corners and try to get it as exact as possible.
3. Once you’re done, select the Track Forward button to let Mocha do the work for you! 

It’ll take a little while, so you can take a break and come back in a few minutes. But what you should notice is that even interactions like the hand moving across the mask won’t interrupt it at all. Because you’ve specified the overall shape of the phone and the planer surface of the screen, the track is going to be incredibly precise and it should stick very well.

Step 4: Re-Import the Data to After Effects

When that’s done, you just need to tell After Effects to use the data that Mocha just collected. To do that, copy and paste your findings into After Effects. 

1. Go to Export > Tracking Data, and choose “After Effects Corner Blur (supporting motion blur)”, and then when you hit copy to clipboard, you’ve basically just copied a bunch of information.
2. Go back into After Effects, and quickly set up a new Solid to double as the screen. Go to New > Solid. 
3. Make it yellow so that it’s really easy to contrast with the blue of the phone.
4. Pre-compose it and label it so we know it’s our screen composition. 
5. Bring the playhead all the way back to the start.
6. Select the Transform properties of your screen composition, and hit Control or Command-V to paste the tracking data to the composition with the yellow solid. 

You should now see the footage with the phone screen replaced with the yellow solid. Play the footage back to see it’s sticking incredibly well, matching the movement of the phone.

Step 5: Repeat the First Steps to Replace the Screen

From here on out the process from the stationary screen is the same, but you don’t have to worry about the jiggling motion now. Just repeat the steps in Part 1.

1. Add your video into the screen layer.
2. Drop the screen layer below the footage layer. 
3. Add a Keylight effect.
4. Key out the blue.
5. Mask around the screen.
6. Set the mask to Keylight.
7. Tweak the blue key.
8. Animate the footage to be a little bit more engaging, making sure to enable motion blur.
9. Top it all off with a lens blur and a little bit of fake screen glare.

---

As you move forward in your video production career, you’ll come across this situation more and more often. Clients will want the option of changing the design of their apps in product videos, and you might discover things in post-production that you didn’t notice when filming.

The solution is to shoot on a green screen and then replacing the device’s screen in After Effects CC. You’ll find it’s the most efficient way to control every detail in your video or film project!