Creating a Procedural Mesh that shows previous frames

Hey Bram!

You're right, you can do this with the Camera Output Texture and Procedural Mesh. If I am understanding you correctly, what you're looking for is to display the Camera Output Texture on a rectangle, with the middle of the rectangle punched out.

(Note: procedural mesh here is useful if you're wanting to create a long path since you do not need to instantiate multiple scene object. but if you only want to create a few, it would be faster to just create a rectangle object, and set an Opacity Texture in the material to punch out the hole (where black is transparent, and white is opaque.) instead of figuring out where a vertex should be). 

Since we are looking to create a triangle, a good place to start is actually in the API documentation page which explains how to create a rectangle and apply a texture on it: 

var left = -.5;
var right = .5;
var top = .5;
var bottom = -.5;

builder.appendVerticesInterleaved([
    // Position         Normal      UV       Index
    left, top, 0,       0, 0, 1,    0, 1,    // 0
    left, bottom, 0,    0, 0, 1,    0, 0,    // 1
    right, bottom, 0,   0, 0, 1,    1, 0,    // 2
    right, top, 0,      0, 0, 1,    1, 1,    // 3
]);

builder.appendIndices([
    0,1,2, // First Triangle
    2,3,0, // Second Triangle
]);

One thing to notice is that the origin of the rectangle that is being drawn is at 0,0,0 since we're setting "left" to be negative, and "right" to be positive. I think it'll be easier if we do math with 0,0 as the bottom left so we don't have to worry about dividing half way. And we'll also set the size to 5 instead of 1 so we don't have to do decimals. 

var size = 5

var left = 0;
var right = size;
var top = size;
var bottom = 0;

So what we need to do next is add vertex points that represents the rectangle we want to punch out inside our current rectangle. To help us with this, we'll define the variable padding which represents how far from the edge of our current rectangle we want our new rectangle to be. (Refer to the diagram below for a visual representation of our code). 

var padding = 1;

Next we'll use this padding to subtract and add from our existing points. So for example, if a point is currently at the left top of the rectangle, the inside rectangle point needs to be shifted more to the right and down (+padding in the x coordinate, and -padding in the y coordinate). (See diagram)

builder.appendVerticesInterleaved([
 // Position(vec3) Normal(vec3) UV(vec2) //vertex index
 left, top, 0, 0, 0, 1, 0, 1, // 0
 left + padding, top - padding, 0, 0, 0, 1, 0,1 // 1

left, bottom, 0, 0, 0, 1, 0, 0, // 2
 left + padding, bottom + padding, 0, 0, 0, 1, 0, 0, // 3

right, bottom, 0, 0, 0, 1, 1, 0, // 4
 right - padding, bottom + padding, 0, 0, 0, 1, 1, 0, // 5

right, top, 0, 0, 0, 1, 1, 1, // 6
 right - padding, top - padding, 0, 0, 0, 1, 1, 1, // 7
]);

Now that we've added our new points, we can create our mesh. Remember that our meshes are made of triangles, so we need to connect our 8 points in 3s. It helps to draw out the points we've added and their vertex index--refer to our image below. 

builder.appendIndices([
 0,1,2, // left frame 1
 1,2,3, // left frame 2
 2,3,5, // bottom frame 1
 2,4,5, // bottom frame 2
 4,5,6, // right frame 1
 5,6,7, // right frame 2
 6,7,1, // top frame 1
 6,0,1, // top frame 2
]);

Now you should see your rectangle with a hole punched out! Since we want our image to be displayed across the rectangle, the last thing we need to do is do something similar for the UV coordinates. Our UV coordinates will tell the Lens what part of an image should be drawn around a point. Unlike our points, however, UV coordinates are normalized. So we need to define a variable for our UV padding. 

var uvPadding = 1 / size;

Finally, in the same way we added and subtracted from our original rectangle to get the inside rectangle, we'll do the same for our UV coordinates. 

builder.appendVerticesInterleaved([
 // Position Normal UV Index
 left, top, 0, 0, 0, 1, 0, 1, // 0
 left + padding, top - padding, 0, 0, 0, 1, 0 + uvPadding, 1 - uvPadding, // 1

left, bottom, 0, 0, 0, 1, 0, 0, // 2
 left + padding, bottom + padding, 0, 0, 0, 1, 0 + uvPadding, 0 + uvPadding, // 3

right, bottom, 0, 0, 0, 1, 1, 0, // 4
 right - padding, bottom + padding, 0, 0, 0, 1, 1 - uvPadding, 0 + uvPadding, // 5

right, top, 0, 0, 0, 1, 1, 1, // 6
 right - padding, top - padding, 0, 0, 0, 1, 1 - uvPadding, 1 - uvPadding, // 7
]);

Now we can set our material to use a texture instead of our vertex color.

In your preview window you should see something like: 

So our final code looks something like:

// -----JS CODE-----
// Builds a quad mesh and applies it to meshVisual
//@input Component.MeshVisual meshVisual
//@input SceneObject cursor
var builder = new MeshBuilder([
 { name: "position", components: 3 },
 { name: "normal", components: 3, normalized: true },
 { name: "texture0", components: 2 },
]);

builder.topology = MeshTopology.Triangles;
builder.indexType = MeshIndexType.UInt16;

var pos = script.cursor.getTransform().getWorldPosition();

var size = 5

var left = 0;
var right = size;
var top = size;
var bottom = 0;

var padding = 1;
var uvPadding = 1 / size;


builder.appendVerticesInterleaved([
 // Position Normal UV Index
 left, top, 0, 0, 0, 1, 0, 1, // 0
 left + padding, top - padding, 0, 0, 0, 1, 0 + uvPadding, 1 - uvPadding, // 1

left, bottom, 0, 0, 0, 1, 0, 0, // 2
 left + padding, bottom + padding, 0, 0, 0, 1, 0 + uvPadding, 0 + uvPadding, // 3

right, bottom, 0, 0, 0, 1, 1, 0, // 4
 right - padding, bottom + padding, 0, 0, 0, 1, 1 - uvPadding, 0 + uvPadding, // 5

right, top, 0, 0, 0, 1, 1, 1, // 6
 right - padding, top - padding, 0, 0, 0, 1, 1 - uvPadding, 1 - uvPadding, // 7
]);

builder.appendIndices([
 0,1,2, // left frame 1
 1,2,3, // left frame 2
 2,3,5, // bottom frame 1
 2,4,5, // bottom frame 2
 4,5,6, // right frame 1
 5,6,7, // right frame 2
 6,7,1, // top frame 1
 6,0,1, // top frame 2
]);


if(builder.isValid()){
 script.meshVisual.mesh = builder.getMesh();
 builder.updateMesh();
}
else{
 print("Mesh data invalid!");
} 

Which roughly translates to: 

Can't wait to see what you make!

Jon

Hey Bram,

This is based of the mesh builder guide you mentioned, so the scene set up is the same.

One difference is that in this guide we haven't implemented how world tracking would affect the square (that is: how the "Cursor" object in the original mesh builder guide would inform the vertices), so we need to turn off the World Tracking component on the Camera. 

Cheers!

Jon