Introduction

Little bit about me

I'm @kosamari on the internet pretty much everywhere except Myspace.

I write JavaScript everyday at Scripto in NYC

I also co-organize JavaScript meetup called BrooklynJS in NYC.

... and my favorite data type is Array

Array and Me

I like dealing with physical array like knit stitches on needle.

I make small programming language that spits out array to design knitting pattern...

Cat pattern by @hamishmurphy

I like writing JavaScript arrays to visualize my knitting process.

GIF from Tumblr

...and turn those array into 0s and 1s to move knitting machine.

I also made D3 scarf !

Because of this, I got really interested in "print out" digital image as knitted fabric. In order to do this, I needed to learn how digital image is perceived by computers... turns out, it's also array of data.

Doing research was reading about a lot of math, & learning to use tools... I wonder, can I do all of this in a language I'm familiar (JavaSript) and a software I already use everyday (browser)?

The answer is YES, let's look at what we can do

What is Canvas

So, let's start with "what is Canvas?"

According to MDN, Canvas was "Added in HTML5, the HTML <canvas> element can be used to draw graphics via scripting in JavaScript. "

In order to talk to this canvas we created, we need to make something called context.

In other wards. creating canvas is like you are reserving piece of land, but you don't know where this land is located yet.

you can set width and height of the element, but we can't really talk to this canvas in JavaScript yet.

So, let's get coding, you can create canvas element like this, or write in your HTML and reference it

You might want to have this land in the city where there is a lot of skyscrapers and very 3d...

... or you might be putting vegetable garden in your back yard.

Talking to canvas

Now that we have context established, we can talk to canvas like draw shapes and fill with color.

We can even load picture into canvas.

There are all human understandable language to communicate with canvas...

...but canvas and code can also talk in data using getImageData() and putImageData()

There are all human understandable language to communicate with canvas, but canvas can also talk to you in data or I call it pixel language.

We are going to look at what ImageData looks like & how to read them in a second, but for now lets recap.

Let's understand ImageData.

So, let's start with "what is Canvas?"

Inside ImageData

So let's look what's inside of ImageData. Few things are somewhat obvious like width and height -it's the width and height of the image, But then you see this array of numbers. This is a flat array of Red, Green, Blue, and Alpha value of each pixel.

What's in a pixel

A pixel in our computer screen consists of Red, Green, and Blue light illuminating at different level, and steps are 0 to 255.

So imagine you have giant single pixel here, combination of 3 lights create color. And if you put equal number for every light you get gray. This is how images are grayscale.

There is 4th number in each pixel called Alpha. This is an extra opacity information so canvas knows how to blend in colors when things are overlapping.

So if we change the 1st 3 numbers to 255, 0, 0, the top left corner pixel change color to Red.

The data is flat, so you need to do a little bit of formatting to get position of each pixel on 2 dimensional plane.

It's like writing on graph paper

I think of it like you have blob of text, and you have write each letter to each to each cell. So the width and height in data gives you size of the graph paper. and you are going to fill in text while breaking line when you run out of grid horizontally.

Dealing with ImageData in your code

When you are dealing with pixel data, you see lots of this double nested for loops. The outer loop moves vertically each row, and inner loop move horizontally to catch each column.

Remember, we have 4 numbers for each pixel, so you'll need to x4 the index to access every data.

...or make an phantom image.

it is 1x1 pixel

with lots of box-shadow

...or make an phantom image.

Pixel Playground

But what about changing appearance of an image ? Let's look at what kind of stuff we can do with data. In short, you apply polynomial function, but hearing that math language freeze my brain...

I like to think of it as shape of a slide in this pixel playground.

origial

So this is your regular slide, or the function. X axis is the input value and Y axis is the output value. If you have input value of 200, this the output value will be 200

invert

You can invert this (quite literally) to get inverted color image. So if I put value of 20 the inverted out put will be 235.

blightness

Going back to original graph (or slide), You can lift the whole thing up and each pixel gets brighter, or darker.

contrasting

You can change slope of this slide to change contrast of a color. In low contrast setting like this, you have narrower range of colors coming out of this process, if you turn in opposite way, you have high range of color (or contrast between highest value and lowest value) coming out.

posterize

If you make the line into steps. You get posterized effect. In this case each red, green and blue lights have 5 possible output which limits the out put color to 125 colors.

solarize

You can make it zigzag, resulting only part of color to be inverted. This makes output image gives unique effect.

threshold

If your input is already gray scaled, and the graph is only 2 steps, you get threshold hold image like this.

Pseudo Color

Lastly you can use different shape of graph for each color channel.The example shown created rainbow color table, which I know is not a great choice to color your visualization, because it's hard to see the steps especially in green part, but you can see why if you have it in graph format. Bigger range of green is on full brightness compare to blue and red.

Touch Perfect

Touch Perfect

Touch Perfect

Touch Perfect

Touch Perfect

...or make an phantom image.

What the heck is convolution?

So now we know how to play with color of an image, but what about changing a structure of image, like blur and sharp. One way to achieve this is called kernel convolution.

But again, math makes my brain freeze, like to think of it as "Pixel Social Graph"

So let's look at how to use pixel social graph, here you have a pixel.

and friends surrounding the pixel.

then you decide how much this pixel in center, also called kernel, get influenced by friends. In very simple blur, it's like you want to blend in with your friends, so you weight each friend equally.

you get average color of the pixel

let's look at how to apply this social graph to an image

Here our target pixel is in top left corner, and I marked size of this social circle in color.

we talked about each pixel has 3 colors

so let's look a just red value for this.

and green

and blue

and you can finally get color of a single pixel

you are going to do that for each color, and each pixel moving the area one pixcel at at time.

Box Blur

You do that by moving one pixel at a time, and you get blur-ed image.

Gaussian Blur

Or you might want to get more influence from your closest friends, you change the number a bit and get little more defined blur.

Sharpen

Sharpening is opposite you do not want to be like your friend, so you rate them negatively while amplifying your self.

tilt

Packaging for the web

Performance

## Make it faster

Doing math on full color, large scake image takes lots of computing power. You can, optimize how you calculate values by tweaking your code, but also you can get help from browser features to help with speed, with Web Workers.

worker is like ISS

worker code.

worker code 1.

worker code2.

worker code3.

worker code 4.

worker sample

## Make it faster

Doing math on full color, large scake image takes lots of computing power. You can, optimize how you calculate values by tweaking your code, but also you can get help from browser features to help with speed, with Web Workers.

toDataURL

toDataURL

toDataURL

toBlob

files

files

files

files

files

fileformat

compression

translucent

grafi

Thanks!