Tamas Piros 0:07
And welcome, everyone. Good evening, good morning, or good afternoon, wherever you may be. And thank you very much for, for joining us. And thank you very much for this great introduction. It’s a pleasure to be here. So today, we’re going to talk about web assembly and the JavaScript. And the title of this talk is, you know, supercharge your JavaScript with with web assembly or with Wesam. And what we’re going to do today is go through my set of slides I have about, you know, 20 slides or so.

So I’m not going to just show you slides because I have a couple of examples of real projects that I worked on that use web assembly and use JavaScript, and I’m going to showcase those to you as well. I got a very good introduction. So I’m not going to reintroduce myself to you. Instead, I thought that we’re going to start with a joke. So the question is, why do and I’ve heard you laughing? This is going to be a terrible joke, by the way. So why do you program is leave that job. And the reason why they leave their job is because they don’t get a race. Now, if you don’t get the joke, you know, a race a race. Now, at this point, I wish I could see your face. You may, please don’t quit. I promise. This is the last joke, right? So very much. We’re going to talk about web assembly and cool stuff, I promise you. Okay, so on a more serious note. If you think about the web platform, and if you think about the web platform, as it was back, you know, two, three or three years ago, and in 2018. This the sort of execution engines that you had in the browser, were these two things you have, you know, a bunch of web API’s, you know, things like the Fetch API and all the other API notification API. And you know, there’s at least 20, if not more web API’s you can use in JavaScript in the browser.

And browsers also contain a virtual machine. And in, you know, up until 2018, the the virtual machine inside the browser was able to execute JavaScript code, okay, that’s, you know, a very sort of simple view of what the browser is doing. Right. So a virtual machine to execute JavaScript code, and then you can access the web API’s. Now, we all know how, you know the web is progressing. And you know, the stuff that comes out is really great. So we essentially went from utilizing servers, to utilizing the browser, a lot more, in fact, in today’s web ecosystem, and in today, when you create all your web applications, you use the browser as your main execution engine. And of course, with that, you use JavaScript. And it fits very well into this ecosystem.

So there’s, you know, vanilla, JavaScript and a bunch of JavaScript frameworks that developers are using. But JavaScript has certain limits. First of all, it was never meant to be used so heavily, you know, it has certain features that are really great for the web, but there are certain functionalities that you can’t do in a performant way using JavaScript. So it’s relatively difficult to achieve low level tasks, you know, inside your browser, using JavaScript without paying a performance penalty, right, there are things that you can do, you know, low level processing, computationally heavy processing, you can do it. And there are ways that you can make that work happen in a performant way, like, you know, use web workers and other stuff.

But JavaScript is not for that. And you may still pay a performance penalty for that. So that’s why people started to think about another way to bring low level code execution to the web platform. And in light of that, back in 2015, this thing called Web assembly was created. Now, there’s one slight note on this slide is that there was this thing called asm.js, which predates WebAssembly by two years, I think ESM. The GS was created by the researchers at Mozilla, if I remember correctly, but essentially that, you know, ASM. JS allows developers to run applications created using the C language as a web application. So effectively taking something that was written in a relatively low level language like C and run it on the web in the browser. And that is really, you know, the father of WebAssembly because WebAssembly allows us to do exactly that. And as of 2019, December of 2019, web assembly made it to be a W3C recommendation, right. So it has a proper spec, you can go to W3C website, you can read it. And web assembly is now part of the web ecosystem.

So what is web assembly? So I do have the sort of official explanation here, which I took from from MDN. Mozilla’s website. And it states that WebAssembly is a low level assembly like language with a compact binary format that runs with a near native performance and provides languages such as C, C++ and rust with a compilation target so that they can run on the web.

That’s a mouthful, isn’t it? That’s this very good, but a relatively complex explanation. So I came up with my version. And that’s run native apps on the web, right. So basically, take your C application to your take a C++ application on your Ross application, and just run it on the web, run it in your browser. That’s essentially what we’re talking about here when we talk about web assembly. And the great thing about web assembly, and because it’s part of the this whole web ecosystem, the wave of assembly was created is that it can access JavaScript we can talk to, or it can communicate with JavaScript. So you have web assembly functions that can be exposed to JavaScript, right? So you can create, you know, a C++ application, or you have an existing C++ application. It has functions, it has methods that you want to access on the web, you can expose those for JavaScript, and you can just invoke them, and a representative will execute dos.

And there’s one sort of slight misconception that I’ve heard a bunch of times, you know, when I talk to people who are WebAssembly, and they say, you know, WebAssembly, is going to replace JavaScript. And that’s not true. WebAssembly is wasn’t created to replace JavaScript at all. It is going to enhance it or it is actually enhancing is augmenting it writes is giving additional power to JavaScript that JavaScript doesn’t have, right. So it allows us to do low level code execution, something that JavaScript is not really good at. And now JavaScript can actually talk to WebAssembly to do that low level code execution.

So if you take a look at the web platform, as it is today, browsers, these are, you know, modern browsers that we’re talking about here. They still have the web API’s. But they do have two separate virtual machines, they have actually made one virtual machine. I’m sorry, I’m not exactly sure about this point at this moment. But there’s this virtual machine that we discussed, which is able to execute JavaScript code. But there’s also a virtual machine that is now capable of executing web assembly code. Okay, and that is now part of the browser. And because of this setup, and this is, you know, a very simple way to show the architecture of a browser, of course. But what I’m trying to highlight here is that because you have the JavaScript code execution, and WebAssembly code execution, happening in the browser, those things can communicate with each other, those things can interact with each other, right? There’s no, there’s sort of no boundaries, that would separate these things that would allow that wouldn’t allow you to make that connection between the two.

There’s also this thing called the WebAssembly JavaScript API, which essentially allows you to do a couple of things, right. So the JavaScript API for WebAssembly means that you can actually work with a web assembly file, and you can do things like load them, or you can create memory and table instances. And effectively, when you work with a web assembly module, you can think of those as just a standard ES6 or ES2015 module, which you import into your application that represent JavaScript API works using promises, you can just reload your web assembly. And then once you do that, you can access the web assembly memory, there’s a file system that you can access, which is going to open up all sorts of very interesting possibilities for you. And then you can just access all the functions from web assembly inside JavaScript is really, really simple. So what that means in terms of, you know, creating a web assembly file, which by the way, has the dot wasm. extension. So that’s why I have this slide titled as process of creating a web. So that’s a wasm file, which is a web assembly file effectively through a web assembly module even.

So the process of this is you create your application and that sort of two ways that you can do this either, you know, write something from scratch in C C++, or as I say, in any other LLVM supported language. So LLVM is is a low level virtual machine. Effectively, it’s a compiler infrastructure. And there are other programming languages that use that, you know, C sharp Haskell Objective C and others. So you either write something in C or C++, for example. And then you use this tool called Emscripten. And you specify your compilation target to be web assembly, it will generate that web assembly file for you. And then you take that WebAssembly finding loaded and consume that using JavaScript in the browser. The other sort of way to do this is that you take something that already existed, so you take an existing C or C++ project, and you want if you want to bring that to the web, you can run it through Emscripten compilation, target wasm. And then you do the loading and consumption of that.

Now, there are of course, other programming languages that you can use to create web assembly modules, I’m oversimplifies. So you could create what we would call non LLVM supported languages. So languages like dotnet, Java, Ruby, go, etc. All of these languages can now compile to web assembly as well. So you write your code using one of these languages, we take an already existing application that you’ve wrote using these languages, you compile it to web assembly, and the process is going to be the same Hello isn’t consuming to write JavaScript. And the list doesn’t stop here, because there are a vast, excuse me, a vast, vast number of programming languages that compile to WebAssembly. So dotnet, C, C++, we mentioned, oh, C#, D, F#, go Java, PHP, Python. And there’s TypeScript as well. So believe it or not, there’s this thing called, I think it’s called assembly script, which allows you to write very strict and typesafe type scripts. And then you can create web assembly from that as well, this person, or maybe there’s an organization, I’m not sure I can’t remember now, that collected a list of languages have compiled to web assembly, so you can take a look at that link. And in that represent tree, you have a list.

And it’s not only a list of languages that do support of Assembly, it also gives you information, whether it has full support, partial support, or support, you know, coming soon or in progress. Okay, so that’s a very brief introduction to web assembly and what you can do, but let me demo a couple of sort of applications that I created using web assembly, and I think we have about four of them that I’m going to go through, there’s going to be two that I’m going to demonstrate in the beginning, very basic applications just to show you the sort of process of creating web assembly files and then to how to consume them using JavaScript. And then the other two applications are going to be slightly more complex. And those are applications that are, you know, I’m using and there’s, you know, some other people are using that for very specific reasons. Okay, So demo time. Let’s go to my editor. In the chat, you can let me know if you think this font size is you know, too small. So just let me know read it as this I think this you know, this is like a six times zoom. So I’m hoping people can can read it, or mark is asking no mention of Rust. Rust is another one, I didn’t list all of the languages because that would have been, you know, just a full slide with all the languages. Rust is is against something that compiles to WebAssembly. Okay, so there’s, you know, rust, go take a look at that list. And it’s likely that you’re going to find your programming language of choice that right.

So two very simple examples that I want to show first one is square.c. Now, you know, quick disclaimer here. What I’m trying to show you here is how to create a WebAssembly file and how to consume that on the web using JavaScript in the browser. I know that JavaScript can do mathematical operations like this, okay, that’s that’s not the point. The point is we have a C function and we’re going to invoke this C function using JavaScript in the browser, okay, but I wanted to keep things as simple as possible. So here we have a very simple C files squared.c. And I mentioned this tool called Emscripten. Emscripten will allow us to take a C or C++ application and specify the compilation target to be web assembly. And in order to do that, I am including the Emscripten headers, I added this keepalive line here. And then we just specify, you know, the simplest C function that we have, which is in square, it takes an integer, and it will return the square value as a squared number.

Now, in order to create a web assembly file, what I had to do was execute this particular command that you see here, I’m not going to execute it now, because I’ve already done it before this workshop, do you notice that I’m running emcc, which is the Emscripten compiler is something that you need to set up. It’s very well documented, if you use Google for Emscripten, you will be taken to I think, probably emscripten.com, or something similar, it’s very easy to set it up. Once you set it up, you will be able to execute this emcc command as well. So what I do is, you know, emcc stated compiler specify the C file, I say web assembly is our target. I can also do extra exported runtime methods. So I could do the C wrap. I could do additional exported methods, or I could specify additional exported methods here. And then what I do is dash O square dor JS. And I’m going to tell you what this credit just does in just a second.

So if you execute this command, you’re going to end up having two separate files, there’s going to be a square that JS and a squared.wasm file. So this is your web assembly module. And then you have this square that Jas, who which looks like a very sort of, you know, complicated JavaScript. But this is the JavaScript file that I will need to load inside my web application. So let’s take a look at how I can bring in the square, you know, web assembly module to index of HTML or to into my browser. So notice that what I have in my HTML, I’m going to open this, open this in my browser in just a moment. But what I have here is an input box with an ID number. So this is where we’re going to, as I’m sure you can guess add a number, then I have a button, Id calculates if we hit that button, I would expect the result of that calculation to come back to this result paragraph. Okay, very simple. So I’m including square.js I’m including the generated JavaScript file in here. And then I have access because I added square the JS I have access to this thing called the module, which is, as part of the Emscripten process square.JS exports a module, which then has access to my web assembly module, which I can then call in my JavaScript in the browser. I then say on runtime initialize, I’m basically initializing my web assembly file.

And what I’ve done. And there are many ways to do this. But this I think, for demonstration purposes, it’s the easiest, I take, or I create an object called API, I call or I create a property on that API object called Calculate square. And I basically specify module dot c rap now remember, C wrapper, something that I exported as part of the web assembly creation. And what this allows me to do is to specify three parameters. The first one is the C function that I would like to call, which is int underscore square. And then I need to specify the return type. And I always mix up these two parameters. So the first one is either the return type for the method here. And then the second parameter is the argument. datatype, right. So if you remember, squared, C, expects an integer and will return an integer. Okay, so I’m just specifying that is number. And that’s a number as well. So return type and argument, or maybe it’s the other way around, I can’t remember now, maybe you can. It is no help in Visual Studio code for this. Okay.

So that’s that. And then I go ahead and voice and the JavaScript right, so I basically call get element by ID calculate on the button. I then add a click event listener for the button. I grabbed the value as well from the input box. So this is a standard JavaScript and then I just call API dot calculate square, and I pass in the input number. So effectively take that number from the input box and I pass it into this calculate square method that I have on my API object. And that is, in turn, going to call in square, and it’s going to pass in the input number as a parameter, and it’s going to return us a number.

So let’s run this. It’s really not, you know, demonstrating the true capabilities and value of web assembly. But you know, once you get an application like this, right, and you understand how to, you know, bring in us how to create a sci fi how to transform that to WebAssembly, how to load that web assembly, how to load the functions and how to code those, you know, a more complex application is going to be, it’s going to be very easy for you to do that. So we have calculate, to calculate and then we get four. So this four is actually coming back from the C in square methods. And that’s, that’s pretty much it. Okay, there’s there’s no magic.

But now, calculate, doesn’t call anything in JavaScript, per se, it is invoking the calculation from WebAssembly. As I said, in a simple, simple example. Now, the other example that I have here is called cube.go. And I’m sure you can guess what this is going to do. Cube, and you will see some differences in here is the C example was relatively straightforward. They’ll go example, in the way how go creates web assembly. It’s, it’s not my favorite. But you know, it’s worth mentioning as well. So if you know go, then this is going to be very, sort of straightforward for you to understand. But what happens here is, first of all, I have an extra import for Cisco slash GS. So we’re going to be needing this, if we want to create a web assembly file from this go file, then I create a cube function.

And notice that I have access to J S value and inputs, which is a JS value array. And this is going to be important because I can directly pass in parameters from JavaScript to these cube methods, I have read the number, so it’s essentially going to accept a number. And I’m going to, you know, cube that number. And that’s what I’m going to return. But look at how I’m returning it. I’m actually from my Go Code, I’m calling document, get element by ID result, and I’m setting the innerHTML to be the result. So I’m basically it I am directly communicating with my document object model. From my go script. Of course, this is good, but you know, ultimately, we’re going to create web assembly. So it’s going to be a web assembly file that will communicate with our top. And then I have a main command. And I then expose the cube function and make it available for JavaScript. Now you see, this is a slightly different way than how the the C version, in inverted commas version worked. But still, it’s going to result in the same behavior. Now in order for us to create the WebAssembly file from this go file, you need to run this. So we need to set the architecture to be wasm. And then we just go go build. And then we create. In this case, I called the Five cube wizard. Now, what’s interesting is that if you run this command, again, you’re going to receive two files. First of all, you’re going to get cubes or bison, as you see here on the on the left hand side, but this is also going to produce wasm on the school act sector chests are in fact, the only thing I can’t remember if this is going to be, it’s maybe something that you need to get from the Go website. And I just had it here, I can’t recall this one on top of my head. But this resume execution file this present executor, JS is something that you will need to add to your HTML. And this is going to allow you to execute the cube doc wasm file that you created from go So this file is just specific to the Go programming language.

So let’s see how this works in this file called index dash go dot HTML. So again, I add wasn’t that excellent and this time Around the loading is slightly different. So we are going to instantiate the web assembly source with these. Actually, if I remember correctly, this is a, this is something that I added because I think Safari didn’t support this version. So this is a workaround to make sure that look, the website file loads in every every browser in the same way. Now, the reason why I can do cause go equals new go, is because I have wasm, execute the GS in here. So that’s going to allow us to instantiate the effectively a simplified version of the Go runtime. Then I call web assembly dot instantiate streaming.

Now what you could do as well, as you know, make sure that web assembly is available for your browser, so you know, the classic checks that you can do, I use the Fetch API, to then fetch the web assembly file. And as I said, this is a promise based API. So this is the WebAssembly JavaScript API that you see here. And then I am going to create this module, create an instance and it’s called called Run, I’m going to run by instance. So if I run my instance, it means that everything that I created in my go file, which now compile down to web assembly, will be accessible for me in here. And what I do have an input box, again, I have a button here. And this time around, I didn’t add an separate script for the onClick handler. For the click handler. Instead, I’m using onclick as an attribute here. And notice, I don’t need to do anything, I can just access the function that I created in ghosts, it’s called cube. And effectively, because I am exporting it here, I can access it inside my HTML inside JavaScript inside anywhere, and I pass in document or get element by ID number, dot value. And notice that I’m not setting anything for the result, because remember, I am setting the result directly from goal. So this is almost like a two way communication where I can access the dough from go.

But because I’m exporting, exposing an inverted commas, something here, I can also call the cube function from JavaScript prices is two way communication happening between these two files. And so this because the same server I wonder where it’s going to place it now, right here, okay. So I need to go to slash index go that HTML. So I get my Calculate button again. So if I try to calculate eight, so that this is now cubing, right, so that’s 27. So it’s three times three times three. Again, the result and the calculation is not done in JavaScript, it is done by the underlying web assembly module is done by using go. So these are some very similar, very simple examples, very straightforward. You can do cubes, and you can do square and you can square numbers using JavaScript just as fine. I’m not trying to advocate that you should use you know, web assembly for for simple things like this. But let me show you two other projects that I had a lot of fun working with, which are going to be slightly more complex. And again, I have two examples for you.

So somebody’s saying that okay, I’m hoping I was a person, Claudia, Claudia de Silva. I hope you can see my screen. If not, as suggested, do you refresh your browser please? Okay, so two more examples. So I just need to open to those. Okay, here’s what how should I do it? Should I show you what it does? Or should I explain it to you? Let me explain this to you first. Now, let me show it to being to deserve to get I have two examples, two more complex examples one using go the other one using C++ actually. So let me bring this over to this screen right here. So I created this tool and just called me check. So you know, I think it was mentioned that this particular talk is sponsored by cloudinary. I also work for Cloudinary. And let me do a quick demo, because I want you to understand what I’m going to show you.

So for those of you don’t know, cloudinary is a cloud based media management solution. So it means you can take your images and videos uploaded, uploaded to cloudinary. Cloudinary will generate you an access URL. And you can then utilize cloudinary optimization features transformation features and their CDN to distribute the media assets. So just to show you a quick example, let’s imagine that we have this URL. Let’s do it this way, split this into two half so that you see what the actual URL is. So this is the URL that you have in my browser.

So we assuming we took this picture uploaded the cloudinary, and we got the URL to access this, if you take a look at the network panel, we will see that this particular file is let’s move this a little bit up. So this particular file is 583 kilobytes in size. And as expected, it is a JPEG. But I find this firefish way to be great. It’s it’s large in dimensions and slightly larger and content length. So what cloudinary can do, there’s a lot of things that Cloudinary can do. But let me show you this one thing F underscore auto.

So first of all, if you work with images, then images on the web, that is you know that there’s JPEG files that you know, that’s PNGs, you know that there’s web P, for example, where p is a very particular image format, because, you know, Google created that and develop the image format, meaning that the Chrome browser supports it. And it’s a very efficient image encoding, but you try to open a web in Safari, it’s not going to do anything because Safari doesn’t support web page. That’s about to change. But you know, the majority of Safari versions out there will not open a web be however, Safari is aware of the JPEG 2000 encoding. So theoretically, what we’re saying here is that for an ideal optimized experience for your users, in terms of performance, you should generate a webpage for Chrome if you generate a JPEG 2000 for Safari and the jpg for everyone else.

Now, with this F auto flag, what clouding your allows you to do, and I’m going to copy this and paste in here is that cloud is going to take a look at your browser, and it will say, well, using Chrome, so I’m going to on the server side, generate a web P and send that web P for you. Because, you know, I can effectively encode that much more efficiently. So in fact, if I just hit enter here, and I What did I do, in May image, acquisition replace sorry, it’s here, F underscore two, you will see that we went from 583 kilobytes this to just 50 kilobytes, purely because this file is now a web p file. And we have features like q underscore auto, which will take a look at every single image that you pass into it, it’s going to analyze the image and reduce the quality of the image without losing what we call visual fidelity. So effectively, what I’m saying is that the image is going to still look the same for the human eye. Okay, so we do this test. So this is now 49 kilobytes, we load this new URL, the size goes down to 32 kilobytes. We haven’t even touched the dimensions, but we save a lot of kilobytes over the wire. So this is what Cloudinary does. At least, you know, this is about 1% of what Cloudinary is doing. But how is this relevant to what we’re talking about here?

So I basically built this tool, because I wanted to know, not only the fact that the image that I get back from cloudinary is a JPEG or it’s a PNG, or it’s a webpage. I wanted to further analyze the image, I wanted to know what version of png we’re talking about, does it? Does it have interlacing? Does it have an alpha channel? You know, if it’s a web p image, which kind of weaponry container, are we talking about? VP8, VP8L. And there’s, you know, there’s lots of differences in between. So basically, I wanted to do lower level analysis of the images that are being returned from Cloudflare.

And because I wanted to do some low level image processing, I really, you know, I probably could have done it in JavaScript, but then, you know, are their clients client side JavaScript libraries available that would allow you to sort of read the first couple of bytes of a PNG and analyze it. I don’t know, maybe there are, but I went down with the web assembly. So what this tool allows me to do. And of course, you know, you could do this with any other image CDN, probably, or you could analyze your images as well, I just hard coded this first bit of the cloud in the URL. And then we have this, you know, the last couple of bits, and this is, you know, an F auto on this particular jpg file. I have the user agents, I’m using Chrome this time around. So if I hit check, and this is a JPEG image, here are the dimensions and I also grabbed the sort of quality of the image, you know, what, what is the quality of this image? You do need to take this value with a pinch of salt, because it’s not always accurate. You know, what if I now add Kyoto? Will things change around? Well, let’s see. And we’re going to do check, well, the size went down.

You know, that’s great. What if I change the dimensions of this image with 500, for example? Well notice if we shrink the image, so w underscore 500, we’ll create a render a with 500 pixel image, and image with a width of 500 pixels. I can’t speak English today, I’m sorry. And notice that the file type that we get back from cloudinary, now changed, so it is now rendering a web UI, which is of a VP eight container, the size is 14 kilobytes, the dimensions are of course, you know, 500, by whatever, because that’s what we set.

This is using a lossy encoding, and it doesn’t have an alpha channel, meaning there are no transparent bits in this particular web page. Great. So what if, you know, what would clouding your render? If I were to use Safari, now, I have this trick for you. Because I’m sure you know that you can’t change the user agent programmatically. But if you use Chrome, and if you use the dev tools, there’s this wonderful thing called network conditions, which will if you uncheck the select Automatically checkbox allow you to set any user agent and is going to mock and pretends to be a safari Mac, and it’s going to send this particular user agent. So notice this user agent updated here as well, I don’t know how much you paid attention to this, but this is now the Safari, you know, Mac OS user agent. So let’s run check again. Notice that this is now a JPEG of 50 kilobytes quality 77 Okay, so completely different response, because I changed the user agent. I did some tests with my other image.

So it was a JPEG I think this should return us on safari, a JPEG 2000. Right? So it allows me to analyze my image further. So just saying it’s a JPEG is telling me that this is JPEG 2000. And I have some more information on things like PNGs. So I have a you’re going to laugh now, but this is this is for real. Darth Vader, the PNG, which is actually an image about Darth Vader. Long story. And so if I hit check, notice that I’m not just saying that it’s a PNG, I’m actually telling what specific type of png we’re talking about. So the different types of PNG, PNG 832 24, I think this is a PNG 32 It has a bit depth of eight is true color with an alpha channel, and it’s not using interlacing. If I change things around, so maybe let’s add f underscore auto. Now hit f4 to decide, well, actually, a JPEG 2000 is much more suitable for displaying this particular image for Mac OS X. You know, I could change to I don’t know Microsoft Internet Explorer 11. In that case, okay. Let’s ignore that. It’s probably my tool that doesn’t work that export it. But if you take a look at Chrome, this is now again a different font. This is now a web p with a V p 8x. container, which has an alpha channel. Anyway, you get the idea, right? See, I’m basically analyzing the image here. And this actually allowed us to to do some some debugging so there was this issue with we’re not really sure a question from from one of our customers. They had this particular image All right here, which is I should probably open that image. And the basically apply, and you don’t see the URL again. So do apologize for that.

So this is the URL. And what they do, they set a, some colors on this image, okay, and they do have some some other transformations in here, which we’re not going to get into. But what happens was that based on the cue auto value, the colors of the image changed. Okay, so when it was rendering PNG, it changed when it was rendering the JPEG, it changed. And with the help of this tool, we were able to figure out that the reason why this color changed is because q ortho decided to generate different versions of PNGs. So sometimes in more colors, sometimes either, with less colors with a smaller color palette. And this color is some very tiny color differences between a Kyoto and another Kyoto version of the image.

But anyway, you get the idea of what we can do you hear now in terms of the code, and I want to crack on with this, because I have another example to show you. So in terms of the code, the way I’ve have constructed this is I have a separate sort of I don’t know how to call this even a package or they call a package and it goes to a separate package for checking jpg separate package for checking png separate package for checking webpages. You know, should I want to maybe in the future check for Avi F files, I can just create a new package here and use that. So the web be check, what’s the plastic of the PNG check. So the PNG check. I don’t even know where to show the most important bit here. Where is the bit that I’ve read the image. So So basically, using go, I can take the image data buffers, I can take the buffer of the image, feed that in here, and I can run it through, for example, this PNG that should say version, so png version, that’s interesting png version methods. So pass in the buffer of the PNG, I read the 23rd and 24th value, what is going to tell me the bit depth what it’s going to tell me the color type. And now I can compare these so if the bit depth is eight, and the color type is three, then it is labeled as a PNG eight. And I use the PNG specification to come up with this right. So this PNG, jpg 24, PNG 32, the difference is that, you know, they would contain more colors and different color color.

So do you know something similar for webP, I’ve read all the information, the low level information from from webP to determine, you know whether the rep p image has an alpha channel, meaning there are transparent parts of it, what kind of, you know, rep the container we’re talking about. And I’m not going to get into that. But if you’re interested, you can take a look at the two webP specification. I put everything together in this file called main.go. I call, you know, everything in here. And what you’ve seen in the cube.go file, everything is, you know, similar to this, except it’s just more code. But effectively, as you can see, I’m setting a response tax, I’m directly typing in HTML in here. And that gets read inside my HTML here, because I fetch the web assembly file, and I can now call I can call this invoke button, passing the cloud in the URL, and the user agent that invokes my

methods from this file called invoke. So that’s what I’m exposing here. And various invoke, right here, invoking invoke, right, so I pass in the URL pass in the the user agent, and I just read the buffer of the image and do all the processing that I have to so I’m not going to go through this in in depth but I hope you kind of get the idea of of what I’m what I’m doing here. So that’s one example. And the other example because I promised you two is a projects that took me many long days and nights to to to convert. It’s quite similar Correct. So similar Chrome was a project that a couple of colleagues of mine at cloudinary has have created back in 2017. So there’s this colleague of mine, Yan, who is an image researcher. So he does. You know, he writes specifications for jpg now.

So he’s, you know, really doing some interesting thing with images. And he created this C++ file called simulacra, which is structural similarity, unveiling locrian, compression rated artifacts. So that’s, that’s a very, it’s a mouthful. But what it means it’s a tool that is capable of comparing two images, and it’s going to return you a number, the closer that number is to zero, the more similar the images are. And it’s a very great tool to compare visual deficiencies of images. So you have, you know, maybe an image that’s very low quality, and you have the image with very high quality, you expect those two images to have a very large number. So it’s going to be like zero point 83. Whereas if the images are closer to zero, so maybe the similar curve result is 0.003, it means those images are very, very close to each other. And the reason why we created this tool, or why he created this tool was to, to be able to justify how our Kyoto works, how this automatic quality reduction works. And similar crud actually took real user data into consideration.

So it’s not just a computational algorithm. We did some research back in 2017, we asked, I don’t know how many hundreds of users, we asked them to compare two images, we took the result of that. And this similar create a CPP file was born. I’m not going to go through this because personally, I don’t understand half of this. But what matters is that this was just a common line tool. Right? So you could do you could compile this C++ project. And you could run it from the command line. And that’s it. And I thought, hang on, wouldn’t this be amazing, as a web assembly file and run it on the internet, so that I can compare images? And so I decided to do run this project, or to create this project? Let me tell you, it was quite a challenge, because there were some changes that I had to change this in this file. Some because it was referencing some older headers, but eventually I managed to make it work. I run it through Emscripten, because this is now C++ already through Emscripten. And oh, my God, I’m running out of time.

Do I do I get five more minutes? I’m sorry, I didn’t realize that as Yes, sorry. I’m just going to show you what it does. So I run it through Emscripten. It generated me this similar crypto JavaScript file, it created me this similar wasm file. And what’s really interesting is that with the WebAssembly JavaScript API, I can actually write files to this to the virtual machines file system. And if I write files, I can write write files to their file system from WebAssembly. And I can read that as well. So what this particular project is doing is taking those two images is going to grab them, it’s going to read, write them on disk is going to run the similar calculation, and it’s going to return a result and add that results to HTML. Again, slightly more complex code. But effectively, this is very similar to the squared C approach that I was showing you before. So let’s check this. So let’s run this even. And the look and feel is going to be very similar. Let’s maximize this, right, so I have this image here. And again, I’m just using carbon right, so this is the black car that we saw before. Very nice image, very crisp, and then I’m using Q one, which is going to generate a quality one, the lowest quality possible for an image and I don’t know how much this is coming through the stream. But this is an awful looking good, you know, it’s pixelated, it’s, you know, it’s clearly a very low quality image.

Now, if I hit Calculate the similarity it does take a couple of seconds because there’s a lot of processing and calculation that needs to take place is 0.4. So this is a very large number meaning that these to me Chances are that they are similar to each other because it’s the same. But because of the quality reduction, we, you know, it’s very bad. That’s what this is effectively saying. So if I remove, you know, q1, and I say, actually, if I just compare the two images, we’ll get a zero because effectively, these are the two images to have the same image with you. And if I do que se 90, so the quality 90 for the JPEG, I’m going to get something like 0.00 Something something says 0.006. Right. So this is a very small number. Actually, a massive I’m sorry, the closer the number is to one, the worse the closer to zero, the better. Okay, so this is very, very close to zero, meaning that these images are very similar to each other. And what we use, or what we do with this tool is actually test our cue to feature whether Kyoto is going to reduce the quality of the image in a way that it’s not going to affect the human eye. Okay. And we calculate the similarity using Kyoto, which is going to pick the most appropriate quality value is 0.03. That’s more than fine, right? It’s still very close to zero. And users will not be affected by this quality reduction.

This is what we effectively say, Okay, so the interesting thing about this project was that I managed to take something that was purposefully written for the CLI using C++. And I managed to port it to the web using WebAssembly. And, admittedly, you know, a lot of work and as part of this process, you know, I discovered, but you know, things like, hey, I can actually write stuff to this virtual file system. And I can read stuff from it’s very, very useful. And that file system is available for JavaScript and for web assembly, as well for it to read. It’s super interesting. Okay, we had the demo time. So I’m going to wrap this up. And I think there’s a lot of questions in the chat as well. I’m hoping that there’s a way that we can distribute the slides for you, you know, a bunch of links for you, you know, Emscripten, there’s a lot of good stuff on MDS website, you know, skoosh dot app, if you follow what Google is doing. So it’s crucial app is actually using WebAssembly, as well for for doing sort of image transformations and optimization in the browser. They took that to the next level, because it also acts as a progressive web app. So for those of you are familiar with progressive web apps, because you have a WebAssembly, file a dot wasm file, effectively, you can cache that. So you can actually create an application that uses web assembly, and it works offline.

So that’s again, you know, opens up a lot of possibilities for you. And I really liked this article, or this case study from Google, where they compared this thing or the hot path, using grep assembly where they compare something created, I think it seems maybe rust or C++ compiled down to WebAssembly, and the same stuff created in JavaScript. And they have this really nice performance comparison between this. And that you can read the case that there. So bunch of links for you. Wasn’t by example also has, I think someone ported and I want to say do or one of these, you know, classic games, written in C++ or whatever, compiled it down to web assembly. And now you can just play it on the web in your processes. Super interesting stuff. Right there. Okay. So thank you. That’s all what I had for you. I hope you enjoyed this presentation. And now, let’s see your questions. I don’t know how you want to do this.

Suz Shardlow 54:05
So talk to us. I’ll go through the questions for you. Because you did answer a couple of them as you went along. So first question, if you go to Developer Tools, can you see the connection between WebAssembly and the JavaScript? Or Are you only going to see the JavaScript that’s been generated?

Tamas Piros 54:25
A very quick answer for you. So there was a latest conference from Google Chrome Dev Summit. So if you take a look at the latest Chrome Dev Summit, there’s a talk by I’m forgetting the name of the person one of the Googlers. The latest Chrome Dev Tools, added additional debugging and discovery features for WebAssembly. Have a look at his talk. He actually tells you how you can see the original represent the original code that you do generate WebAssembly from in Dev Tools or something very similar. So there’s lots of capabilities that you can do. So you know, connections, you can do debugging. So you can do live debugging, using dev tools for your web assembly, you can do tracing, you know, all sorts of stuff. Take a look at that about a half an hour talk. Unfortunately, I’m forgetting the guy’s name says in the latest Chrome Dev Summit. If you go to YouTube, just do you know, Chrome Dev Summit, 2020, or 2020 20. And do representative you’ll find it.

Suz Shardlow 55:31
Cool. Okay, so that answers another question. Somebody’s asked about the debugging capability. So go and have a look at this Chrome Dev Summit and search for the web assembly talk, which is about half an hour long. Cool. Thanks for that. Can we do async functions via web assembly?

Tamas Piros 55:51
I assume async functions from the JavaScript side. So I’m not sure what what you’re asking here. But you can so the the WebAssembly JavaScript API is probably space. So instead of using them, you could use async await? If that’s your question.

Suz Shardlow 56:13
Cool. If that isn’t the question, please come back into the chat and clarify that. And they’ve asked another question, if you need to process a huge object inside the function inside web assembly, is there a transfer overhead like with web workers?

Tamas Piros 56:31
I suppose so depends on the size of the object, of course. I don’t have experience with that. So I can only do some guesswork here, I would assume you know, the larger the file is robably. There’s also things that you can do, when you and I’ve done this with Emscripten, you can extend the allocated memory for your WebAssembly. file. So by default, when you don’t specify anything, Emscripten will just you know, allocate 25 megabytes of memory, this is a true understanding, as an example. But there’s an option when you compile your C or C++ to WebAssembly, to say, well actually allow me to use X amount of memory. So that would help but again, this would be all dependent on the size of your, of the object that we’re talking about here, you know, many, many gigabytes. You know, that will take some time to process as well. You can also leverage this this file system that I mentioned. So there are ways but I don’t have personally have any experience with with larger files on that, you know, a couple of maybe four or five megabytes images, and those were working relatively fast.

Suz Shardlow 57:58
Cool. Again, on a similar note, somebody’s asking, is there any overhead in accessing the thumb from web assembly in comparison to accessing it using JavaScript directly?

Tamas Piros 58:11
Again, I haven’t experienced any noticeable delay. The one thing I would add with go and and web assembly, because because that’s the place where probably this question came up where I was showing you that from go, you can access the dome. And you know, don’t take it the wrong way. I do like go and I do like the purpose some of the features. What I don’t like is that they bring the entire sort of like a minified version of the Go runtime to your browser. It that is the overhead in my, in my view, right? It is the JavaScript that you need to include is very, very large. So use some caution there. But again, I don’t have experience with or comparable experience with you know, hey, I’m accessing the dome using this way and or accessing the dome that way and whether they perform differently.

Suz Shardlow 59:09
Cool. Okay. I think it’s very much a case of have a play with it and see, see what you find, isn’t it? Somebody else is asking how we deal with concurrency in WebAssembly?

Tamas Piros 59:25
That’s actually a good question. I did read there was this article was so when I tried to find an article for you, and I’m going to update the Resources section, there was this person who was cornered birding. One of his one of his projects with the company where he was working. I think it was a Dutch Museum, museums website. So they had this website where you could walk in the museum and they would run the you know, very large, three hundreds 60 degree images in the browser using JavaScript and something else. And then slowly they transition into using WebAssembly. And there’s this very lengthy article about what they tried and, you know, problems with concurrency and using Emscripten. And using these flags, and that didn’t work. So there are some options there. So I’ll remember to add that link for you in the resources to to have a look at that.

Suz Shardlow 1:00:28
Cool, thanks for that. So you gave us some really good examples using images, have you got any use cases or examples that are not to deal with processing images?

Tamas Piros 1:00:43
If you so my recommendation would be if I just go back on my still share, and just have a look at this wisdom by example page. Or in fact, and I can, I can go here. Now. If I can click this, please go there. Okay. You will find all sorts of examples on the site, not only just just images, select your languages, you know, Ross, assembly script, etc. There are some examples in here graphics working with audio. Well, that’s it for for rust, tiny go. Okay, it’s essentially still graphics and audio. So there are some examples in here. But if you just Google, you know, what are some use cases, examples projects, you will find some additional ones, you know, as I said, Someone converted one of the games. I think it was do what may be Super Mario, I can’t remember now. They ported that that’s not image, as you saw, as you know, work with audio, where we, you know, the types of graphics are just images. There’s, there’s all sorts of examples. And there was one more. So I can do some, do some research that there are other use cases for web assembly, other than just images, I did use images, because you know, I work at a company that deals a lot with images. But there are some other examples out there.

Suz Shardlow 1:02:22
Cool. And the last question has to do with performance again, so somebody’s asking if the browser’s going to use more resources, if you’re using WebAssembly.

Tamas Piros 1:02:34
From my experience, it’s minimal. I also recommend that you read this case study that I’m linking to on the slides that I’m sharing at the moment. That explains I think, memory consumption as well, in terms of web assembly, so you can you can actually follow that article. And, and try to implement some of those suggestions that they say so you know, maybe have a web assembly file already. So you can implement those. And you can do some some comparison on your own again, because and I think I had this question before as well. Can you run web assembly web workers as well? I think the answer is yes. So you can always sort of look at web assembly out from your main thread and do the execution somewhere else. So in that sense, again, your performance should not be impacted. And also this article that I mentioned with this guy during the transformation of this website, he does share as far as I remember some very specific details in terms of performance. And you know how, when he used Emscripten, generated the web assembly, this is how long it took. This is how much memory it was using, etc. And then he I think, ultimately went down to us whether the route of using the TypeScript based wisdom, so assembly script, and then he shares his insights in terms of performance on there as well.

Suz Shardlow 1:03:58
Cool. Thanks, Ian. Some folks have put some examples in the chat, which was really helpful for them. So yeah, they were quite challenging this week, for some reason, but thanks for your time to demo all that without so I really like the way you started off with a very simple example and then then build it up. So hopefully, we’ll see you again soon here at CFE.

Tamas Piros 1:04:23
Thank you for having me.