How do you define a color?

August 1, 2017, 7:00 am

≫ Next: The brightest crayon in the shed

≪ Previous: Is 1.0 delta E a "just noticeable difference"?

$

0

0

I got an interesting question from a good buddy of mine, Mitchell Vaughn, Well, I kinda exaggerate when I say good buddy, cuz I just met him. And it was online, so maybe it doesn't count? But, he said he liked my blog, so I think that's the foundation for a lifelong friendship. Yes. I am that vain.

Here is the question:

I hope you don't mind me asking you a question, which I imagine is a loaded question...but here it is: Are L* a* b* coordinates a color's undeniable "definition"? In other words, is there anything else that needs to be in place to define a color...mathematically speaking? I realize there are several measuring guidelines that need to be met like light source and angle, etc. but wanted to get your thoughts on this. 

Thank you, Mitchell

I have three answers, the first one simple and theoretical, the second one complicated and theoretical, and the third one practical.

Quick answer

Color is properly defined as a sensation inside our head. So, once we have defined the relative amounts of light that the three cones in the eye will see, the color has been defined. Well, almost. The eye, brain, and the glop in between need a reference point to establish what white is. All color understanding in the brain is compared against this white reference. But since you're talking about L*a*b* values, this has already been mixed into the soup.

Sealab stew is a hearty meal all by itself!

So, the first answer is that, yes, an L*a*b* value defines a color, provided you know what white is.

Necessary qualifications

But when we are talking about L*a*b*, we are almost always talking about the color of objects -- be it the ink on a package, the paint on a wall, or the color of a plastic part. And (OK, this is gonna sound weird) objects don't have colors.

Consider the red ace of hearts. What color is the heart? Red, of course.

I took three pictures of two aces below. The camera and cards were not moved, all I did was change the lighting. Honest to god... there was no Photoshopping in the images below. No special tricks, other than playing with the lighting.

What color is the ace of hearts?

In the image at the left, taken with "normal" lighting, we see "normal" colors. The heart on the ace of hearts is red. For the middle image, I turned off all the lights in the room and illuminated the cards only with a 456 nm blue LED. The color of the red ace of hearts is now pretty much the same as the ace of clubs; it's black.

The right-most image shows what happened when I swapped in a 626 nm red LED instead of the blue LED. Now the color of the red ace of hearts is white. Or maybe it's red?  I dunno how you would explain it. True statement: The color of the red heart is nearly the same as the color of the card stock. Subjective statements: If you call the card stock white, then the heart is white. If you call the card stock red, then the heart is also red.

I will pause while you consider the implications of this. The color of the heart depends on whether your brain has decided that the card stock is white or red.

This is an extreme example, but all objects, to a greater or lesser extent, will change color as the spectral characteristics of the light changes. I might add, two colors may match under one illumination, but not under another. The ace of hearts matches the ace of clubs at the blue light club, but matches the card stock in the red light district. My wife loves to say the word for that: metamerism. She is not all that fond of saying red light district, or any of the other words for that.

To define the color of an object, we need to specify the spectral characteristics of the light that hits the sample. 

To make matters worse, the amount and spectral composition of light that reflects from an object depends to a greater or lesser extent on the angle that the light hits, and the angle from which it is viewed.

The images below are of the same blackberry, with the same camera and camera position, but with different lighting. The image on the left has a point source of light, and the image on then left shows the blackberry illuminated by diffuse lighting. The colors of corresponding parts of the two images are not the same.

Which blackberry looks the most succulent?

To define the color of an object, we need to specify the angles of illumination and of viewing. There are an infinite number of combinations, but a small collection of combinations have been standardized so that we can actually communicate about color values. The most common choices are 45/0 geometry (which is equivalent to 0/45) and diffuse geometry.

Am I done yet? No. Our perception of color depends (slightly) on whether it is a small object (projected onto just the inner circle of the retina, called the fovea) or a larger object (which extends to more of the retina). The relative concentrations of cones are different in the fovea than the rest of the retina, so our perception of color changes.

To define the color of an object, we need to specify whether the object is small (the 2 degree observer) or larger (the 10 degree observer). In case you are not confused enough yet, I discuss standard illuminants and observers in a blog post called How many D65s are there in a 2 degree observer?

In summary, the color of an object is a property of the object itsewlf, but also of the spectral composition of the incident light, the angles of incidence and viewing, and the size of the object. Based on that, once you have specified the L*a*b* value and all of these conditions (by saying, for example, 45/0 geometry, D50 illumination, 2 degree observer), you have defined the color sensation, and the color of the object has been defined.

So the second answer is that, for an L*a*b* value to have a precise meaning, you have to specify the instrument geometry, the illuminant, and the observer (2 or 10 degree).

Note that this does not mean the object won't have a different color under different conditions. Sorry for the double negative. Lemme try again. Objects in the mirror may appear closer than they are. Product is measured by weight and not volume some settling may have occurred during shipping. No warranties are express or implied. And, the color of your tie and sport coat may not match under the funky mood lighting when you get back to your apartment.

Practical answer

There is another important definition for anyone in the business of making stuff that has a specified color. Color is defined as that thing that the customer is willing to pay you for, provided you get it correct. It is whatever is defined in the contract. Without a contract detailed enough to have teeth, the correct color is whatever the customer likes.

The astute print buyer will recognize that his Wheaties package might be sitting on a shelf right next to another Wheaties package that was printed in a different press run or even at a different plant. The astute print buyer will recognize that an off-color package (just like an off-color joke) runs the risk of sitting on the shelf until expiration date, at which time it will get thrown out, much to the dismay of everyone who hates to see good Wheaties go bad.

This astute print buyer will also recognize that metamerism could be an issue if different sets of pigments are used to create the ink on the package. In that case, the astute print buyer might see fit to define the color in terms of spectral values, or in terms of color specifications under multiple illuminants.

So, all those previous answers are just academic if you live in the real world and want to get paid for your print job!

The standards folks, I might add, are pushing for a spectral definition of colors. Various tools are being put into place to allow the standardized communication of desired spectra.

---

The brightest crayon in the shed

August 8, 2017, 7:00 am

≫ Next: Seven incredible duct tape life hacks

≪ Previous: How do you define a color?

$

0

0

People are always telling me that I am not just the brightest crayon in the shed. But which crayon is?

The yellow crayon screams out "Pick Me! Pick Me!"

Well, white is the logical answer, but yellow is pretty darn close to white in terms of brightness. And a very bright yellow can also be very saturated. In this sense, yellow is kind of an anomaly in the color kingdom. All other colors, when they get saturated (color scientist use the term high chroma), get darker.

Why is yellow such a gosh darn bright color?

Munsell agrees

I am not just making up this "yellow is a bright color" thing. Munsell agrees with me, as we can see from the Munsell color pages below, where I have circled (or ellipsed in some cases) the most saturated colors on each page of constant hue.

A selection of Munsell plates with constant hue

Some preliminary stuff

I will explain why yellow is such a gosh darn bright color, but first, I need to get some fundamentals in place.

The Cohans

Those of you who are fans of my blog (I think there are currently seven of you, worldwide) will no doubt remember a stirring blog post I wrote about the cones in the eye. The image below is a recap of the exciting opening premise of that blog, suggesting that the eye has three types of color sensors, and that they are red, green, and blue.

I looked deep into her eyes,
and suddenly and inexplicably found myself hungry for Haagen Dazs

The excitement generated at the start of the blog post was short-lived. The whole point of the post was that the colors of the cones in the eye were not quite as black-and-white as the first guess of red, green, and blue. But, if you are taking the final exam for Color Theory 101, then red, green, and blue is the correct answer. Red/green/blue is also suitable for our purposes.

Definition of eight basic colors

The RGB Cohans in the eye (not to be confused with G. M. Cohan, who was red, white, and blue) lead to a simple explanation of the eight basic colors in Color Theory 101. This is all based on a lie, but it is a useful lie. If the red cone is the only cone that sees the light, then the color we will perceive is red. Similarly, if the incoming light stimulates only the green cones, then we see green. And guess what? If it is only the blue cones, then we see blue. I bet you had already guessed that one.

How about combinations? If red and blue cones are activated (but not the green cones) then we see magenta. If the activated cones are the blue and green ones (but not the red), then we see cyan. Finally, if blue is left out and the red and green cones get all the attention, then we see yellow.

I said finally, but really I didn't mean finally, since there are two more combinations. We see black when all the cones are inactive, and white when all three are activated.

The table below summarizes the cone responses to each of the eight basic colors in the RGB color system.

Color	Red cones?	Green cones?	Blue cones?
Black	No	No	No
Red	Yes	No	No
Green	No	Yes	No
Blue	No	No	Yes
Yellow	Yes	Yes	No
Magenta	Yes	No	Yes
Cyan	No	Yes	Yes
White	Yes	Yes	Yes

Oh... I forgot to mention... these eight colors are all the strongest colors. The cells in the table above that have "No" in them mean "zero light", and the ones with "Yes" in them mean full intensity. There are a zillion combinations where the amount of light captured by the three cones is somewhere between full on and full off. These are not the strongest colors.

The importance of the lightness channel

There is a famous experiment -- very famous, everyone has heard of it -- where an ace was flashed on a screen for an instant. If that instant is really small, then the subjects had no trouble identifying the object as the ace of spades. But if they slowed it down so that the ace stayed on the screen for just a little longer, people got all kinda cornfoozled. When the researcher extended the time just a bit more, then the subjects could readily understand that they were being shown a red ace of spades. (For those who did not grow up in a casino, the ace of spades is supposed to be black, not red.)

Here is a YouTube version of the red spade experiment

This dorky (but famous) little experiment sheds a little light on how our eye/brain works, more particularly on how the color signals are encoded in the neurons that connect the eye to the brain. There is one signal (carried on a neuron) which transmits our perception of brightness. 

This is a special signal. It arrives to the brain quicker than the other signals. When the red ace is flashed quickly enough, the signals that further narrow the color down to red don't make it to the brain in time for analysis. A little longer flash, and the red signal makes it, but the signal isn't stable enough for full pattern recognition. A little longer still, and the brain has time to parse the image out and understand the weirdness.

Not only does the brightness signal show up first, but it is far more important than the other signals in terms of our understanding the scene. I am old enough to remember complaining bitterly about being the absolute last family in the whole town to get a color TV. Well, maybe not the last, but my buddy, Gary, had a color TV well before we did. His father worked for Motorola. My father gave me the lame excuse that black and white were colors, so our TV was actually a color TV. It's a wonder that I can function as an adult at all, what with the extreme deprivation and subsequent emotional trauma that I was subjected to!

People leading colorful lives, despite living in a black and white world

The funny thing about B&W television is that it actually worked. I don't recall my father ever setting me down and explaining that light gray could mean the taupe uniforms of Andy and Barney, or it could mean a Caucasian skin tone, or it could mean Ethel's blond hair. Somehow, I just subconsciously understood the color transform, and never questioned it. At least until I enviously watched Gary's TV.

So, why is saturated yellow so bright?

We now have enough background to explain the enigma of bright yellow. One sentence brings it all together: the brightness signal which is fed to the brain from the eye is a combination of the signals from the red and green cones. There are two separate signals that encode 1) the difference between green cones and red cones, and 2) the difference between blue cones and green cones.

All colors that have red and green at the same intensity have the same brightness. A quick look at the table shows that white and yellow are the only colors where red and green are at full intensity.

And that is why yellow is such a bright color.

Seven incredible duct tape life hacks

August 15, 2017, 7:00 am

≫ Next: Intellectual humor

≪ Previous: The brightest crayon in the shed

$

0

0

I have assembled seven of my favorite life hacks for the guy who always has a roll of duct tape handy.

Hack #1 - Ugly mug

Let's face it. There are many of us who are just too ugly to stomach looking at ourselves in the mirror every morning. Imagine how much better you would feel if you saw Brad Pitt looking back at you? A pic of your favorite actor, a little duct tape, and you can wake up feeling sexy!

Hack #2 - Broken mirror?

Are you in the middle of seven years' bad luck? And can't scrape up the money to replace that mirror? Just duct tape your cell phone to the mirror and put it in selfie mode!

Hack #3 - Read both sides of a newspaper

A drop of oil and you can cut your newspaper reading time in half! Ok, maybe it's not duct tape, but what real man doesn't always have a can of WD40 handy?

Hack #4 - Cell phone mute

You feel a sneeze coming on. You know that your cell phone has a mute button somewhere, but don't have the time to find the manual and look it up. Grab a strip of duct tape, and viola! You got a mute button. (BTW, did you tell your wife that you are at a "convention"? You can also use this on the camera lens when you Facetime with her.)

Hack #5 - Screen dimmer

Those darn cell phones never seem to get that whole auto-brightness figgered out. Duct tape + old man sunglasses = easy reading!

Hack #6 - Can't figger out Word?

Let's face it. Microsoft Word is just too complicated! Two strips of duct tape and a piece of paper and you are word processing with the pros!

Hack #7 - Pill storage

Doncha just hate those cumbersome, ugly, hard-to-open pill boxes? A piece of duct tape and a wall is all you need to organize your pills!

Impressed? Look for my new book in quality hardware stores everywhere.

Intellectual humor

August 24, 2017, 8:00 am

≫ Next: Interpreting color difference data - a practical discussion of the CRF

≪ Previous: Seven incredible duct tape life hacks

$

0

0

And now for a little intellectual humor...

For my wordie friends

For my science groupies

Any history buffs out there?

Interpreting color difference data - a practical discussion of the CRF

September 6, 2017, 9:13 am

≫ Next: Statistical process control of color, approaching a method that works

≪ Previous: Intellectual humor

$

0

0

My avid readers (yes, both of you) will realize that I have been on a mission, a holy quest, for process control of color. The holy grail that I seek is a technique for looking at the color data of a color-making process, and distinguishing between normal and abnormal variations in the process. This is subtly different from setting acceptance tolerances. The first is inward looking, seeking to improve the process. The second is outward-focused, with the ultimate goal of getting paid by a happy customer.

I'm not talking about getting paid. Unfortunately, getting paid is rarely the outcome of this blog!

In this blog post, I talk about a tool for color geeks. This tool is appropriate for analysis of whether the color is acceptable. I will discuss whether this tool is acceptable for identifying changes in the process. I promise to write about this tool for customer satisfaction later.

Recap of exciting events leading up to this

Just to recap, I wrote a lengthy and boring sequence of blogs on statistical process control of color difference data. The four part series was aptly called Statistical process control of color difference data. (Note that the link is to the first part of the series. For those who tend toward masochism, each of the first three posts have a link to the next in the series, so feel free to indulge.)

The topic for today's blog post is a tool from statistics called the CPDF (cumulative probability density function). At least that's the name that it was given in the stats class that I flunked out of in college. It is also called CPD (cumulative probability distribution), CDF (cumulative distribution function), and in some circles it's affectionately known as Clyde. In the graphic arts standards committee clique, it has gone by the name of CRF (cumulative relative frequency).

I blogged on the CPDF/CPD/CDF/Clyde/CRF before, and by golly, I just realized that I used two of the names in that blog post. This violates rule number 47 in technical writing, which states that you have to be absolutely consistent in your use of technical terms. This is also rule number 7 in patent writing, where it is generally assumed that different words mean different things. I will try to be consistent in calling this tool the CRF. Please forgive me if I slip up and call it Clyde.

Now that the nomenclature is out of the way, today's blog is an extension of the previous blog post on CRF. Today, I want to discuss the practical aspects. Looking at a CRF, what can we discern about the nature of a collection of color differences? And importantly, what conclusions should we avoid coming to, however tempting they may be.

Brief refresher

Below we have a picture of a CRF. The horizontal axis is color difference. The purplish blue curve represents the percentage of color differences that are smaller than that particular DE.

Picture of Clyde from another blog post of mine

It is rather easy from this plot to determine what are called the rank order statistics. The red arrows show that the median (AKA the 50th percentile) is a shade under 1.5 DE. The green arrows show the 95th percentile as being 3.0 DE.

So, one use of the CRF is to quickly visualize whatever percentile is your favorite. You like the 68th percentile? No problem. Your neighbor's dog prefers the 83rd? Yup, it's there as well.

The plot above is real data from one set of real measurements of real printed sheets. The plot below shows nine different sets of real data. The different data sets show medians of anywhere from 0.3 DE to 4 DE. It's clear from looking at the plots that some are tighter distributions than others. Most people would say that the tighter ones are better than the broader ones, but I try not to be so judgmental. I try to love all CRFs equally.

Picture of Bonnie, from the same blog post, who exhibits multiple personality disorder

So, we see that another use of the CRF is to visualize the overall magnitude of a set of color differences. Of course, the median or 95th percentile also give you that impression, but the CRF plot is visual (great for people who are visual learners), and it incorporates all the percentiles into one picture.

Do we need more than one number?

This begs a question. If I know the median, do I need the 95th percentile as well? Is there any additional information in having both numbers?

I assessed this in that prior blog post that I keep referring back to. Based on the data sets that I had at my disposal, I found that there is a very strong correlation between the median and the 95th percentile (r-squared = 0.903). You could get a pretty good guess at the 95th percentile just by multiplying the median by 1.840. That's actually good news for those who have only a small amount of data to draw conclusions from. The median is a lot more stable than trying to infer the median from (say) 20 points.

But I should add a caveat -- the data that I was analyzing was pretty much all well-behaved. If the data is not well behaved, then the ratio between the two will probably not be close to 1.840. So, having both numbers might provide exactly the information that you are looking for. I mean, why would you be wasting your time looking at color difference data, if not to identify funkiness???!?

The next graph illustrates this point. I started with that same set of 9 CRFs from the previous graph. I scaled each of them horizontally so that they had a median color difference of 2.4 DE. If all the different percentiles have the same basic information, then all the curves would lay right atop one another. 

But they don't all lay atop each other. After doing the rubber band thing to adjust for scaling, they don't all have the same shape. The orange plot is below the others in the lower percentiles, and mostly above the others at higher percentiles. The red plot is kinda the opposite.

What does this observation about the two CRFs tell us about the DE values from which the CRFs were created? And more importantly, what does this tell us about the set of color differences that went into creating the CRF?

(That was a rhetorical question. Please don't answer it. If you answered it, it would steal the thunder from the whole rest of this blog post. And that would make me sad, considering the amount of time that I am going to put into analyzing data and the writing this post! You may actually learn something, cuz I think that no one in the history of the known universe has investigated this topic to the depth that I did to write this post.)

The simple answer to the rhetorical question is that the orange plot is closer to having color difference values that are all the same, and the red plot has more of a range of color difference values. But the answer is actually more nuanced than that. (I recently heard the word nuanced, and I admit that I have been waiting for the opportunity to show it off. It's such a pretentious word!)

Here is the third thing we learn by looking at the CRF: Not all CRFs are created equal. The shape of the CRF tells us something about the color difference data, but we aren't quite sure what it tells us. Yet.

Looking at cases

To help build our intuition, I will look at a few basic cases. I will show the distribution of points in an a*b* plot, and then look at the associated CRF. Conclusions will be drawn and we will find ourselves with a deeper appreciation of the merits and limitations of CRFology as applied to color difference data.

About the data

The data that I use for the different cases is all random data generated deep in the CPU of my computer. I used a random number generator set up to give me DL*, Da* and Db* data that is normally distributed (Gaussian). Note that while DE values are definitely not normally distributed, the variations in the individual components of L*a*b* are more likely to follow a normal distribution, at least when the color process is under "good control". Whatever that means.

A peek inside my computer as it generated data for CRFs

In most of the cases following, I have looked at data that is strictly two-dimensional (only a* and b* values, with the ridiculous assumption that L* has no variation). I will bring three-dimensional variation in on the last two cases. Those two will blow your mind.

All of this work is done with the 1976 DE formula, simply because it is sooooo much easier to work with. The conclusions that we draw from analysis of these cases will not change if we use the DE2000 color difference formula. This is not immediately evident (at least not right now), but trust me that I kinda have a feeling that what I said was true.

I should mention one other thing. The figures below all show scatter plots and CRF plots. I decided to use 200 data points for the scatter plots, since that made a plot where the scatter was pretty clear. If I went larger, the dots all merge into a blob, which is bad. But worse than that is the fact that the size of the blob depends on the number of points in the scatter plot. (I have a solution to that, but it it several blog posts away).

For the CRF plots, 200 points would give a jaggy appearance that would mask the true underlying shape. So, for the CRFs I chose the luxury of enlisting a few more CPU cycles to generate 10,000 data points. My computer didn't seem to mind. At least it didn't say anything.

In all the scatter plots below, I have plotted Da* and Db*, and not a* and b*. The values are relative to some hypothetical target a*b* value.

Case 1

This first case is the most basic. We assume that the distribution of measurements in a*b* is a circular cloud. The values for Da* and Db* are both normally distributed with standard deviation of 1.0 DE, and Da* and Db* are uncorrelated. You can probably see this; I show the scatter plot in the upper left hand corner of the figure, and the associated CRF below.

Case 1, uncorrelated variation of equal size in a* and b*

The lower right corner of the figure introduces a new metric for analysis of the CRF; the median-to-95th percentile ratio. I intend this to be a parameter which describes the shape of the CRF. Dividing by the median makes it independent of the overall spread of the CRF. I have decided to give this ratio a sexy name: MedTo95. Kinda rolls off the fingertips as I type it. 

Note that a lower value of MedTo95 corresponds to a CRF that has an abrupt transition (high slope at the median), and a lower number indicates a more laid back shape.

(The astute reader will note that the number 1.840 that I described in a previous blog as a way to estimate the 95th percentile from the median was a MedTo95 metric based on real DE data. The really astute reader will note that 1.840 is not equal to 2.11. The really, really astute reader will draw an inference from this inequality. The CRF shown in Case 1 is a bit abnormal.)

Case 2

The most natural question to ask is what happens if the cloud of DE data is moved over a little bit. To put this into context, this is a DE cloud where there is a bias. We have the usual variation in color, but we are also not quite dead-on the target value.

The Figure below shows the Case 1 data in black and this second DE data in green. I thought it was pretty clever of me to come up with a figure like this that is so incredibly intuitive. If you had to scratch your head for a while before catching on, then I guess I am not as clever as I would like to think.

Case 2, which is the same as Case 1, except shifted over

It should come as no surprise that the CRF has been expanded outward toward larger DE values. We have added a second source of color difference, so we expect larger DE values.

Important note for the analysis of the CRFs: The bias in the color (of production versus target a*b*) caused a reduction in MedTo95.

I should also point out that it makes no difference whether the cloud in a*b* has shifted to the left by 2 DE, to the right by 2 DE, or upward or downward. The DE values will all be the same, so the CRF will be the same.

Case 3

It is hard to compare the shape of the black and green plots of Case 2, since one has been stretched. I have altered the cloud of points in the figure below so that the CRFs have the same median. Note that this scaling of the CRF required a commensurate scaling of the a*b* cloud. So, the green cloud is more compact than the black cloud of points. The standard deviations, shown at the bottom left corner of each of the scatter plots, were cut in half.

Case 3 - two data sets with the same median, but different offset

The MedTo95 ratio was 1.70 in Case 1, and is 1.69 in this case -- almost identical. That's reassuring. I mean, that's why I introduced this shape parameter as a ratio.  

Tentative conclusion #1: MedTo95 kinda works as a parameter identifying the shape.

We see that introducing a bias in our process (the color is varying about a color somewhat different than the target color) changed the shape of the CRF. The CRF of the biased data makes a faster transition, that is, it has a higher slope at the median, that is, MedTo95 is smaller.

Tentative conclusion #2: A lower MedTo95 is an indication of bias - that you're not hitting the target color.

(Please note that I underlined and boldfaced the word tentative. This might be foreshadowing or something.)

Case 4

The next most obvious change we could make to the distribution is to change the relative amount of variation in a* and b*, in other words, to make the scatter plot elliptical. This is to be expected in real data. Imagine that the plot below is of the variation in a* and b* measurements of a solid yellow throughout a press run. The predominant cause of variation is the variation in yellow ink film thickness, which is reflected mostly in the b* direction. 

I will take this a step further. The eccentricity (as opposed to circularity) of the scatter plot is an indication that one cause of variation is predominant over the others. 

The figure below shows the effect that a 5:1 aspect ratio has on the CRF.

Case 4 - elliptical variation

This is cool. The shape of the CRF has changed in the opposite direction. I would call this a mellow, laid-back, sixties kind of cool and groovy CRF, one which makes the transition gradually. The slope at the median is smaller. Graciously, MedTo95 has responded to this by getting much larger. Once again, MedTo95 has given us an indication of the shape of the CRF.

Perhaps we can tie the change in MedTo95 back to the variation in a*b*? 

Tentative conclusion #3: A higher value of MedTo95 is an indication of a departure from circularity of the scatter plot. 

Now we're getting somewhere! We can look at MedTo95 and understand something important about the shape of the scatter plot. If we see MedTo95 go up, it is a sign that we have one source of variation which is rearing its ugly head.

But once again, the word tentative is bold faced and underlined. It's almost like I am warning the reader of some broader conclusion that may eclipse this one.

Case 5

Case 4 looked at eccentricity in the b* axis. This is typical of variation in yellow, but magenta (for example) tends to vary a lot more in a*. What if the angle of the eccentricity changes? 

I played with my random number generator to simulate a variation which has been rotated so that the major axis is in the a* direction. I show the figures below.

Case 5 - a comparison of eccentricity in two different directions

This is reassuring. The CRFs are pretty much identical, and a so are the MedTo95 values. This shouldn't be much of a surprise. A moment's consideration should convince one that the color difference values (in DE76) would be the same, so the CRF shouldn't change. This will not be the case for DE2000 values.

This is good news and perhaps not-so-good news. The good news is that the CRF and MedTo95 of DE76 values is irrespective of the predominant direction of the variation. The bad news is that the CRF and MedTo95 of DE76 values are irrespective of the predominant direction of the variation.

Tentative conclusion #4: CRF and MedTo95 don't know nothing from the direction of variation. 

Case 6

We have looked at a bias and eccentricity of variation in isolation. How about if both occur? In the figure below we look at one possible example of that. The blue cloud has been shifted to the right, and squished horizontally. It was also squished just a bit vertically, just so the median is that same as all the other CRFs. 

Case 6, in which we encounter both sliding over and squishing

From the figure, it is clear that the combination of these two effects causes a CRF that is more uptight than the standard scatter plot that we started with. The new CRF is reluctant to change initially, but changes rapidly once it decides to change.

Thus, we re-iterate tentative conclusion #2: A lower MedTo95 is an indication of bias - that you're not hitting the target color. And, of course, we can forget about tentative conclusion #3: A higher value of MedTo95 is an indication of a departure from circularity of the scatter plot.

How does elliptical distribution with offset (Case 6) compare Case 3, where the scatter plot shows a circular distribution with offset? The two are compared in the figure below.

A comparison of two biased distributions

Here we see two CRFs that are pretty darn close if you look at the area above the median. The MedTo95 of the two are also (to no one's surprise) very close. If I may remind you, the CRFs represent a collection of a whopping 10,000 data points where the initial distributions were algorithmically designed to be normal distributions. You ain't never gonna see no real CRF that is as pristine as these CRFs.

Our tentative conclusions are starting to unravel. :(  

Tentative conclusion #5: There ain't no way, no how, that you can use MedTo95 to diagnose ellipticity when there is a bias.

But, these  conclusions are based on what's going on above the median. There is still some hope that the stuff below the median might pan out. We would need, of course, an additional parameter. Maybe MedTo25? 

Case 7

In Case 6, we looked at elliptical variation that is perpendicular to the bias. The bias was off to the right, and the principle axis of the variation was up and down. Let's look at bias and variation that are both along the a* axis. This is shown in the next figure.

Case 7 - comparison of variation parallel and perpendicular to bias 

The new curve - the one in violet - kinda looks like red curve shown in Case 4. Things have certainly gotten complicated! I will try to capture this in another tentative conclusion.

Tentative conclusion #6: In the presence of both elliptical variation and bias, elliptical variation in the direction of the bias looks similar to elliptical variation. Elliptical variation perpendicular to the direction of the bias looks like bias. 

Ummm... I didn't stop to consider what happens when the elliptical variation is at 45 degrees to the bias. Presumably, it looks a lot like circular variation with no bias. That ain't so good. 

I probably should actually show an example of this. I think the CRF of elliptical variation at 45 degrees to direction of the bias would look a lot like the black CRF that we have been using as a reference, at least above the waist. But, rather than head further down the rabbit hole, I have one more consideration.

Case 8

All of the examples so far have made the assumption that the variation is strictly two-dimensional, that is, in a* and b*. That's a simplification that I made in order to aid in our understanding of the interpreting of a CRF. One would expect that three-dimensional variation is more likely to be encountered in the real world.

In the cyan of the figure below, I modeled spherical variation which is of equal magnitude in L*, a*, and b*.

Case 8 - the effect of dimensionality on the CRF

By comparing the cyan CRF to the black (two-dimensional), we see that adding a third dimension has the effect of making the transition sharper and of decreasing MedTo95. The red CRF has been added to suggest the effect of reducing the dimensionality to something closer to one.

(Some readers may be thinking something along the line of "chi-squared distribution with n degrees of freedom, where n is some number between 1 and 3, where that number might not be an integer." If those words are gobbledy-gook, then that's ok.)

This next figure compares the CRF of a three dimensional spherical distribution with a two dimensional circular distribution with a little bias. 

Comparison of spherical distribution with circular distribution with bias

I think that this might be just a tad scary for those who wish to use the CRF to divine something about the scatter of points in color space. We see two distributions that are nothing like each other, but yet have CRFs that are very similar. 

In a theoretical world, one might be able to tell the difference between these two. But, there are two things fighting against us. First, we never have CRF plots that are as clean as the ones I have created. 

Second, this blog post shows that we have a lot of knobs to play with. The shape of the CRF is effected by the length of all three of the axes of the ellipsoid, as well as by the magnitude and direction of the bias with respect to the axes of the ellipsoid. Without a lot of trying, I have twice come up with pairs of dissimilar distributions where the CRFs are similar. And I haven't even considered variations that are non-normal. Given a bit more time, I think I could get some pairs of CRFs that would boggle the mind. 

The non-tentative conclusion

If two CRFs are different, we can pretty definitively make the statement that there is some difference between the distributions of color differences. But, one might just as well look at tea leaves to divine the nature of the difference in the distributions. Furthermore, if the CRFs of two sets of color difference data are similar, one cannot come to any conclusions about the similarity between the underlying color variation in CIELAB space.

This blog post and the exciting conclusion was not an Aha! experience for me. The Aha! moment occurred earlier, when I was coming to grips with the fact that a bias will mask variation in CIELAB. This was described in one of my blog posts about process control of color difference data. 

So, what's the answer? How can we look at a cloud of points in CIELAB and draw any inferences about it? The following diagram is a clue -- drawing ellipsoids that "fit" the variation of points in CIELAB. The ellipses are an extension of the standard deviation to three-dimensions. This involves the co-variance matrix of DL*, Da*, and Db* values. It involves a strange concept of taking the square root of a matrix - not to be confused with the square root of the components of a matrix. And it involves principle component analysis. And it is related to Hotelling's T-squared statistic.

The ellipses below were generated using this technique. I will get around to blogging about this eventually!

I gratefully acknowledge the comments are proofreading from a good friend, Bruce Bayne. He won't admit it, but he is a pretty sharp guy.

Statistical process control of color, approaching a method that works

September 12, 2017, 8:03 pm

≫ Next: Just plain stupid

≪ Previous: Interpreting color difference data - a practical discussion of the CRF

$

0

0

As many of you know, I have been on a mission: to bring the science of statistical process control to the color production process. In my previous blog posts (too numerous to mention) I have wasted a lot of everyone's time describing obvious methods that (if you choose to believe me) don't work as well as we might hope.

Today, I'm going to change course. I will introduce a tool that is actually useful for SPC, although one big caveat will require that I provide a sequel to this interminable series of blog posts.

The key goal for statistical process control

Just as a reminder, the goal of sadististical process control is to rid the world of chaos. But to rid the world of chaos, we must first be able to identify chaos. Process control freaks have a whole bunch of tools for this rolling around in their toolboxes.

The most common tool for identifying chaos is the run-time chart with control limits. Step one is to establish a baseline by analyzing a database of some relevant metric collected over a time period when everything was running hunky-dory. This analysis gives you control limits. When a new measurement is within the established control limits, then you can continue to watch reruns of Get Smart on Amazon Prime or cat videos on YouTube, depending on your preference.

Run-time chart from Binney and Smith

But when a measurement slips out of those control limits, then it's time to stop following the antics of agents 86 and 99 and start running some diagnostics. It's a pretty good bet that something has changed. I described all that statistical process control stuff before, just with a few different jokes. But don't let me dissuade you from reading the previous blog post.

There are a couple other uses for the SPC tool set. If you have a good baseline, you can make changes to the process (new equipment, new work flow, training...) and objectively tell whether the change has improved the process. This is what continuous process improvement is all about.

Another use of the SPC tool set is to ask a pretty fundamental question that is very often ignored: Is my process capable of reliably meeting my customer's specifications?

I would be remiss if I didn't point out the obvious use of SPC. Just admit it, there is nothing quite so hot as someone of your preferred gender saying something like "process capability index".

What SPC is not

Statistical process control is something different from "process control". The whole process control shtick is finding reliable ways to adjust knobs on a manufacturing gizmo to control the outcome. There are algorithms involved, and a lot of process knowledge. Maybe there is a PID control loop, or maybe a highly trained operator has his/her hand on the knob. But that subject is different from SPC.

Statistical process control is also something different from process diagnostics. SPC won't tell you whether you need more mag in your genta. It will let you know that something about the magenta has changed, but immediate job of SPC is not to figger out what changed. This should give the real process engineers some sense of job security!

Quick review of my favorite warnings

I don't want to appear to be a curmudgeon by belaboring the points I have made repeatedly before. (I realize that as of my last birthday, I qualify for curmudgeonhood. I just don't want to appear that way.) But for the readers who have stumbled upon this blog post without the benefit of all my previous tirades, I will give a quick review  of my beef with ΔE based SPC.

Distribution of ΔE is not normal

I would hazard a guess that most SPC enthusiasts are not kept up at night worrying about whether the data that they are dealing with is normally distributed (AKA Gaussian, AKA bell curve). But the assumption of normality underlies practically everything in the SPC toolbox. And ΔE data does not have a normal distribution.

I left warnings about the assumption of abnormality in Mel Brooks' movies

To give an idea of how long I have been on this soapbox, I first blogged about the abnormal distribution of color difference data almost five years ago. And to show that it has never been far from my thoughts and dreams, I blogged about this abnormal distribution again almost a year ago.

A run-time chart of ΔE can be deceiving

Run-time charts can be seen littering the living rooms of every serious SPC-nic. But did you know that using run-time charts of ΔE data can be misleading? A little bit of bias in your color rendition can completely obscure any process variation, lulling you into a false sense of security.

My famous example of a run-time chart with innocuous variation (on upper right)
that hides the ocuous variation in the underlying data (underlying, on lower left)

The Cumulative Probability Density function is obtuse

The ink has barely had a chance to dry on my recent blog post showing that the cumulative relative frequency plot of ΔE values is just lousy as a tool for SPC.

As alluring and seductive as this plot is,
don't mix CRF with SPC!

It can be a useful predictor of whether the customer is going to pay you for the job, but don't try to infer anything about your process from it. Just don't.

A useful quote misattributed to Mark Twain

Everybody complains about the problems with using statistical process control on color difference data, but nobody does anything about it. I know, you think Mark Twain said that, but he never did. Contrary to popular belief, Mark Twain was not much of a color scientist.

The actual quote from Mark Twain

So now it's time for me to leave the road of "if you use these techniques, I won't invite you over to my place for New Year's eve karaoke", and move onto "this approach might work a bit better; what N'Sync song would you like to sing?"

Part of the problem with ΔE is that it is an absolute value. It tells you how far, but not which direction. Another part of the problem is that color is inherently three dimensional, so you need some way to combine three numbers, either explicitly or implicitly.

Three easy pieces

Many practitioners have taken the tack of treating the three axes of color separately. They look at ΔL*, Δa*, and Δb* each in isolation. Since these individual differences can be either positive or negative, they at least have a fighting chance of being somewhat normally distributed. In my vast experience, when a color process is in control, the variations of  ΔL*, Δa*, and Δb* are not far from being normal. I briefly glanced at one data set, and somebody told me something or other about another data set, so this is pretty authoritative.

Let's take an example of real data. The scatter plot below shows the a*b* values of yellow during a production run. These are the a*b* values of two solid yellow patches from each of 102 newspaper printers around the US. This is the data that went into the CGATS TR 002 characterization data set. The red bars are upper and lower control limits for a* and for b*, each computed as the mean, plus and minus three standard deviation units. This presents us with a nice little control limit box.

Scatterplot of a*b* values of solid yellow patches on 102 printers

There is a lot of variation in this data. There is probably more than most color geeks are used to seeing. Why so much? First off, this is newspaper printing, which is on an inexpensive stock, so even within a given printing plant, the variation is fairly high. Second, the printing was done at 102 different printing plants, with little more guidance other than "run your press normally, and try to hit these L*a*b* values".

The variation is much bigger in b* than in a*, by a factor of about 4.5. A directionality like this is to be expected when the variation in color is largely due to a single factor. In this case, that factor is the amount of yellow ink that got squished onto the substrate, and it causes the scatterplot to look a lot like a fat version of the ideal ink trajectory. Note that often, the direction of the scatter is toward and away from neutral gray.

This is actually a great example of SPC. If we draw control limits at 3 standard deviation units away from the mean, then there is a 1 in 200 chance that normally distributed data will fall outside those limits. There are 204 data points in this set, so we would expect something like one data point outside any pair of limits. We got four, which is a bit odd. And the four points are tightly clustered, which is even odder. This kicks in the SPC red flag: it is likely that these four points represent what Deming called "special cause".

I had a look at the source of the data points. Remember when I said that all the printers were in the US? I lied. It turns out that there were two newsprinters from India, each with two representatives of the solid yellow. All the rest of the printers were from either the US or from Canada. I think it is a reasonable guess that there is a slightly different standard formulation for yellow ink in that region of the world. It's not necessarily wrong, it's just different.

I'm gonna call this a triumph of SPC! All I did was look at the numbers and I determined that something was fishy. I didn't know exactly what was fishy, but SPC clued me in that I needed to go look.

Two little hitches

I made a comment a few paragraphs ago, and I haven't gotten any emails from anyone complaining about some points that I blissfully neglected. Here is the comment: "If we draw our control limits at 3 standard deviation units away from the mean, then there is a 1 in 200 chance that normally distributed data will fall outside those limits." There are two subtle issues with this. One makes us over-vigilant, and the other makes us under-vigilant.

Two little hitches with the approach, one up and one down

I absentmindedly forgot to mention that there are two sets of limits in our experiment. There is a 1 in 200 chance of random data wandering outside of one set of limits, and 1 in 200 chance of random data wandering outside of the other set of limits. If we assume that a* and b* are uncorrelated, then this strategy will give us about a 2 in 200 chance of accidentally heading off to investigate random data that is doing nothing more than random data does. Bear in mind that we have only been considering a* and b* - we should also look at L*. So, if we set the control limits to three standard deviation units, then we have a 3 in 200 chance of flagging random data.

So, that's the first hitch. It's not huge, and you could argue that it is largely academic. The value of "three standard deviation units" is somewhat arbitrary. Why not 2.8 or 3.4? The selection of that number has to do with how much tolerance we have for wasting time looking for spurious problems. So we could correct this minor issue by adjusting the cutoff to about 3.15 standard deviation units. Not a big problem.

The second hitch is that we are perhaps being a bit too tolerant of cases where two or more of the values (L*, a*, and b*) are close to the limit. The scatter of data kinda looks like an ellipsoid, so if we call the control limits a box, we are not being suspicious enough of values near the corners. These values that we should be a bit more leery of are shown in pink below. For three dimensional data, we should raise a flag on about half of the regions within the box.

We need to be very suspicious of intruders in the pink area

The math actually exists to fix this second little hitch, and it has been touched on in previous blog posts, in particular this blog post on SPC of color difference data. This math also fixes the problem of the first hitch. Basically, if you scale the data axes appropriately and measure distance from the mean, the statistical distribution is chi-squared with three degrees of freedom.

(If you understood that last sentence, then bully for you. If you didn't get it, then please rest assured that there are at least a couple dozen color science uber-geeks who are shaking their head right now, saying, "Oh. Yeah. Makes sense.")

So, in this example of yellow ink, we would look at the deviations in L*, a*, and a* and normalize them in terms of the standard deviations in each of the directions, and then add them up according to Pythagoras. This square root of the sums of the squares is then compared against a number that was pulled from a table of chi-squared values to determine whether the data point is an outlier. Et voila, or as they say in Great Britain, Bob's your uncle.

Is it worth doing this? I generally live in the void between (on the one side) practical people like engineers and like people who get product out the door, and (on the other side) academics who get their jollies doing academic stuff. That rarified space gives me a unique perspective in answering that question. My answer is a firm "Ehhh.... ". Those who are watching me type can see my shoulders shrugging. So far, maybe, maybe not. But the next section will clear this up.

Are we done?

It would seem that we have solved the basic problem of SPC of color difference data, but is Bob really your uncle? It turns out that yellow was a cleverly chosen example that just happens to work out well. There is a not-so-minor hitch that rears it's ugly head with most other data.

The image below is data from that same data set. It is the L* and b* values of the solid cyan patches. Note that, in this view, there is an obvious correlation between L* deviation and b* variation. (The correlation coefficient is 0.791.) This reflects the simple physics: as you put more cyan ink on the paper, the color gets both more saturated and darker.

This image is not supposed to look like a dreidel

Once again, the upper and lower control limit box has been marked off in dotted red lines. According to the method which has been described so far, everything within this box will be considered "normal variation". (Assuming the a* value is also also within its limits.)

But here, things get pretty icky. The upper left and lower right corners are really, really unlikely to appear under normal variation. I mean really, really, really. Those corner points are around 10 standard deviation units (in the 45 degree direction) from the mean. Did I say really, really, really, really unlikely. Like, I dunno, one chance in about 100 sextillion? I mean, the chance of me getting a phone call from Albert Munsell while giving a keynote at the Munsell Centennial Symposium are greater than that.

Summarizing, the method that has been discussed - individually applying standard one dimensional SPC tools to each of the L*, a*, and b* axes individually - can fail to flag data points that are far outside of the normal variability of color. This happens whenever there is a correlation in the variation between any two of the axes. I have demonstrated with real data that such variation is not unlikely, in fact, it is likely to happen when a single pigment is used to create color at hue angles of 45, 135, 225, or 315 degrees.

What to do?

In the figure above, I also added an ellipse as an alternative control limit. All points within the ellipse are considered normal variation, and all points outside the ellipse are an indication of something weird happening. I would argue that the elliptical control limit is far more accurate than the box.

If we rotated the axes in the L*b* scatter plot of cyan by 44 degrees counter-clockwise, we have an ellipse that is perpendicular to the new horizontal and vertical axes. When we look at the points in this new coordinate system, we have rekindled the beauty that we saw in the yellow scatter plot. We can meaningfully look at the variation in the horizontal direction separately from the variation in the vertical direction. From there, we can do the normalization that I spoke of before and compare against the chi-squared distribution. This gives us the elliptical control limit shown below. (Or ellipsoidal, if we take in all three dimensions.)

It all makes sense if we rotate our head half-way on its side

This technique alleviates hitches #1 and #2, and also fixes the not-so-minor hitch #3. But, this all depends on our ability to come up with a way to rotate the L*a*b* coordinates around so that the ellipsoid lies along the axes. Not a simple problem, but I hear someone in the back of the room whispering "principle component analysis". That technique, tied in with singular value decomposition, and eigenvectors and eigenvalues, can tell us how to rotate the coordinates so that the individual components are all uncorrelated.

Just plain stupid

September 13, 2017, 8:04 am

≫ Next: Can a light be gray?

≪ Previous: Statistical process control of color, approaching a method that works

$

0

0

Just in case you were hoping for another blog post about stupid memes, I can top those last few blogposts!

Can a light be gray?

October 7, 2017, 2:47 pm

≫ Next: Statistics of multi-dimensional data, theory

≪ Previous: Just plain stupid

$

0

0

I follow Quora. I am not saying I am proud of that, but at least I will admit it. I look for questions in my topic of expertise, which is to say, color. And of course, my motives for answering the questions are purely altruistic. Answering the questions is part of my crusade to overcome the general public's ignorance of that noblest of the Sciences, Color Science.

Or maybe I just like to hear myself talk.

Regardless of the reasons, here is a question that I recently answered. My answer has been embellished a bit for this blog post -- mostly to make me sound more important. But I also added some cool pictures.

Is there gray light?

This is an interesting question! I have some experiments that will answer the question.

First experiment

If you connect a white LED to a variable power supply, and gradually turn the voltage down, it will always appear white, even though gray is somewhere between full white and black. I show that in the image below. This is a white LED (color temperature of maybe 6500K) run at 2.4V. This is about as low as it will go without flickering out. The camera clearly sees it as white.

There is no way to suppress the whiteness of this LED

Don't have a variable power supply? You can buy a white LED flashlight and leave it on until the batteries almost run down to nuthin. The LEDs will still be white.

So, first answer: No, there ain't no such thing as gray light. 

Second experiment

But if your arrange those white LEDs into a matrix and call that matrix a computer screen, then you can dim a portion of those white LEDs and get gray. Yes, Virginia, there is gray light, and it's what's coming at you when you look at the image below! Contrary to the guy who wrote about the first experiment, you can make gray light with white LEDs.

You're looking at gray light, right this very minute!!

(I should clarify... some, but not all, computer displays use white LEDs as a backlight behind filters. The idea here is that in principle, you could make a computer display with white LEDs, and you could display gray on that screen.)

Third experiment

Turn your entire computer screen into “gray” (RGB values of 128, for example), and turn out the lights in your room. After a few minutes, you will perceive the screen as white.

I am totally dumbfounded by how white my gray screen looks
Or maybe just dumb? 

Why did that happen? Gray is not a perceivable color all by itself. To see it, you need a white to reference it to.

In the first experiment, the white LED is not putting out a huge amount of light, but the light from the white LED is all coming from a small point. This means that the intensity at that point is very, very high, and likely much brighter than anything else in your field of view.

In the second experiment, I didn’t say this, but it is likely that there are some pixels on your computer screen that are close to white (RGB=255), so the area with RGB=128 will appear gray in comparison. In the third experiment, the only white reference that you have is the computer screen itself, so once your eyes have accommodated to the new lighting, the computer screen will be perceived as white.

Fourth experiment

I came up with a startling way to demonstrate this idea that "gray is perceived only in comparison to a reference white". Brilliant idea, really. I used the same set up I did for that first cool picture of a white LED. But in this case, I used two LEDs, wired in series. Note that I had to crank up the voltage to 4.8V. Electricity passes through each of the LEDs, so in principle they produce the same amount of total light.

The difference between the two LEDs is that the one on the right doesn't have a clear plastic bubble -- the plastic bubble on the LED on the right is a translucent white. The one on the right is a diffuse LED. The light from the diffuse LED is about the same, but it is spread out over a larger area, and not focused, so the amount of light hitting my eye is much less.

My camera saw the diffuse LED as being somewhat dimmer than the one on the left. Maybe from the picture you would call this a gray LED? My eyes looked at the two white LEDs and saw the one on the right as being gray. Honest to God, it was emitting gray light. My eyes saw the LED on the left, and used that as the white reference. The fact that I was drinking heavily during this fourth experiment is largely irrelevant.

Today's special - gray LEDs

So, I can definitively say that "gray" light exists, since I built a system with both a white and a gray LED. I'm sure if I would have introduced this a few weeks ago, I would have gotten an early morning call from Mr. Nobel about some sort of prize. Well, maybe next year. I will try to look surprised.

Moral

This blog has to have a moral. It was a bit hard for me to set up an experiment that demonstrated the emission of gray light. Why? Light, when taken in isolation, can never be gray. We only see gray when it is viewed in contrast to another brighter, whiter color.

---

Statistics of multi-dimensional data, theory

November 14, 2017, 6:00 am

≫ Next: Statistics of multi-dimensional data, example

≪ Previous: Can a light be gray?

$

0

0

This blog post is the culmination of a long series of blog posts on the statistics of color difference data. Most of them just basically said "yeah, normal stats don't work". Lotta help that is, eh? Several blog posts alluded to the fact that I did indeed have a solution. The most recent of which alluded to a method that works in the very title of the blog post: Statistical process control of color, approaching a method that works.

Now it's time to unveil the method.

Generalization of the standard deviation

One way of describing the technique is to call it a generalization of the standard deviation to multiple dimensions -- three dimensions if we are dealing with color data. That's a rather abstract concept, so I will explain.

     One dimensional standard deviation

We can think of our good friends, the standard deviation and mean, as describing a line segment on the number line, as illustrated below. If the data is normally distributed (also called Gaussian, or bell curve), then you would expect that about 68% of the data will fall on the line segment within one standard deviation unit (one sigma) of the mean, 95.45% of the data will fall within two sigma of the mean, and 99.73% of the data will be within three sigma of the mean.

As an aside, note that not all data is normally distributed. This holds true for color difference data, which is the issue that got me started down this path!

So, a one-dimensional standard deviation can be thought of as a line segment that is 2 sigma long, and centered on the mean of the data. It is a one-dimensional summary of all the underlying data.

     Two-dimensional standard deviation

Naturally, a two-dimensional standard deviation is a two-dimensional summary of the underlying two-dimensional data. But instead of a (one-dimensional) line segment, we get an ellipse in two dimensions.

In the simplest case, the two-dimensional standard deviation is a circle (shown in orange below) which is centered on the average of the data points. The circle has a radius of one sigma. If you want to get all mathematical about this, the circle represents a portion of a two-dimensional Gaussian distribution with 39% of the data falling inside the circle, and 61% falling outside.

Two dimensional histogram of a simple set of two dimensional data
The orange line encompasses 39% of the volume.

I slipped a number into that last paragraph that deserves to be underlined: 39%. Back when we were dealing with one-dimensional data, +/- one sigma would encompass 68% of normally distributed data. The number for two-dimensional data is 39%. Toto, I have a feeling we're not in one-dimensional-normal-distribution-ville anymore.

Of course, not all two-dimensional standard deviations are circular like the one in the drawing above. More generally, they will be ellipses. The the length of the semi-major and semi-minor axes of the ellipse are the major and minor standard deviation.

--- Taking a break for a moment

I better stop to review some pesky vocabulary terms. A circle has a radius, which is the distance from the center of the circle to any point on the circle. A circle also has a diameter, which is the distance between opposite points on the circle. The diameter is twice the radius.

When we talk about ellipses, we generally refer to the two axes of the ellipse. The major axis is the longest line segment that goes through the center of the ellipse. The minor axis is the shortest line segment that goes through the center of the ellipse. The lengths of the major and minor axes are essentially the extremes of the diameters of the ellipse. They run perpendicular to each other.

An ellipse, showing off the most gorgeous set of axes I've ever seen

There is no convenient word for the two "radii" of an ellipse. All we have is the inconvenient phrases semi-major axis and semi-major axis. These are half the length of the major and minor axes, respectively.

--- Break over, time to get back to work

The axes of the ellipses won't necessarily be straight up and down and left-to-right on a graph. So, the full description of the two-dimensional standard deviation must include information to identify the orientation of these axes.

The image below shows a set of hypothetical two-dimensional data that has been ellipsified. The red dots are random data that was generated using Mathematica. I asked it to give me 200 normally distributed x data points with a standard deviation of 3, and 200 normally distributed y data points  with a standard deviation of 1. These original data points (the x and y values) were uncorrelated.

This collection of data points were then rotated by 15 degrees so that the new x values had a bit of y in them, and the new y values had a bit of x in them. In other words, there was some correlation (r = 0.6) between the new x and y. I then added 6 to the new x values and 3 to the new y values to move the center of the ellipse. So, the red data points are designed to represent some arbitrary data set that could just happen in real life.

I performed an ellipsification, and have plotted the one, two, and three sigma ellipses (in pink). The major and minor axes of the one sigma ellipse are shown in blue.

Gosh darn! that's purdy!

The result of ellipsifying this data is all the parameters pertaining to the innermost of the ellipses in the image above. This is an ellipse that is centered on {6.11, 3.08}, with major axis of 3.19 and minor axis of 1.00. The ellipse is oriented at 15.8 degrees. These are all rather close to the original parameters that I started with, so I musta done sumthin right.

I also counted the number of data points within the three ellipses. I counted 38.5% in the 1 sigma ellipse, 88.5% in the 2 sigma ellipse, and 99% in the 3 sigma ellipse. (Of course when I say I did this, I really mean that Mathematica gave me a little help.) If the data follows a two-dimensional normal distribution, then the ellipses will encompass 39%, 86.5%, and 98.9% of the data. This is one indication that this condition is met.

The following pieces of information are determined in the ellipsification process of two-dimensional data:

     a) The average of the data which is the center of the ellipse (two numbers, for the horizontal and vertical values)
     b) The orientation of the ellipse (which could be a single number, such as the rotation angle)
     c) The lengths of the semi-major and semi-minor axes of the ellipse (two numbers)

The ellipsification can be described in other ways, but these five numbers will tell me everything about the ellipse. The ellipse is the statistical proxy for the whole data set.

     Three-dimensional standard deviation

The extension to three-dimensional standard deviation is "obvious". (Obvious is a mathematician's way of saying "I don't have the patience to explain this to mere mortals.")

The result of ellipsifying three-dimensional data is the following nine pieces of information that are necessary to describe an arbitrary (three-dimensional) ellipsoid:

    a) The average of the data (three numbers, for the average of the x, y, and z values)
    b) The orientation of the ellipsoid (three numbers defining the direction that the axes point)
    c) The lengths of the semi-major, semi-medial, and semi-minor axes of the ellipse (three numbers)

The image below is an ellipsification of real color data. The data is the color of a solid patch as produced by 102 different newspaper printing presses. There were two samples of this patch from each press, so the number of dots is 204.

The 204 dots were used to compute the three-dimensional standard deviation, which is represented by the three lines. The longest line, the red line, is the major axis of the ellipse, and has a length of 5.6 CIELAB units. The green and blue lines are the medial and minor axes, respectively. They 2.2 and 2.1 CIELAB units long. All three of the axes meet at the mean of all the data points, and all three are two-sigma long (+/-1 sigma from the mean). Depending on the angle you are looking, it may not appear that the axes are all perpendicular to each other, but they are.

Ellipsification of some real data, as shown with the axes of the ellipsoid

Trouble viewing the image above? The image is a .gif image, so you should see it rotating. If it doesn't, then try a different browser, or download it to your computer and view it directly.

What can we do with the ellipsification?

The ellipsification of color data is a three-dimensional version of the standard deviation, so in theory, we can use it for anything that we would use the standard deviation for. The most common use (in the realm of statistical process control) is to decide whether a given data point is an example of typical process variation, or if there is some nefarious agent at work. (Deming would call that a special cause.)

We will see an example of this on real data in the next blog post on the topic.

Statistics of multi-dimensional data, example

November 21, 2017, 6:00 am

≫ Next: Blue skaters

≪ Previous: Statistics of multi-dimensional data, theory

$

0

0

In the previous blog post, Statistics of multi-dimensional data, theory, I introduced a generalization of the standard deviation to three-dimensional data. I called it ellipsification. In this blog post I am going to apply this ellipsification thing to real data to demonstrate the application to statistical process control of color.

I posted this cuz there just aren't enough trolls on the internet

Is the data normal?

In traditional SPC, the assumption is almost always made that the underlying variation is normally distributed. (This assumption is rarely challenged, so we blithely use the hammers that are conveniently in our toolbox -- standard SPC tools -- to drive in screws. But that's another rant.)

The question of normalcy is worth addressing. First off, since I am at least pretending to be a math guy, I should at least pay lip service to stuff that has to do with math. Second, we are venturing into uncharted territory, so it pays to be cautious. Third, we already have a warning that deltaE color difference is not normal. Ok, maybe a bunch of warnings. Mostly from me.

I demonstrate in the next section that my selected data set can be transformed into another data set with components that are uncorrelated, have zero mean and standard deviation of 1.0, and which give every indication of being normal. So, one could us this transform on the color data and apply traditional SPC techniques on the individual components, but you will see that I take this one step further.

    Original data

I use the solid magenta data from the data set that I describe below in the section below called "Provenance of the data". I picked magenta because it is well known that it has a "hook". In other words, as you increase pigment level or ink film thickness, it changes hue. The thicker the magenta ink, the redder it goes. Note that this can be seen in the far left graph as a tilt to the ellipsoid.

I show three views of the data below. The black ellipses are slices through the middle of the ellipsification in the a*b* plane, the L*a* plane, and the L*b* plane, respectively.

View from above

View from the b* axis

View from the a* axis

    Standardized data

Recall for the moment when you were in Stats 201. I know that probably brings up memories of that cute guy or girl that sat in the third row, but that's not what I am talking about. I am talking about standardizing the data to create a Z score. You subtracted the mean and then divided by the standard deviation so that the standardized data set has zero mean, and standard deviation of 1.0.

I will do the same standardization, but generalized to multiple dimensions. One change, though. I need an extra step to rotate the axes of the ellipsoid so that all the axes are aligned with the coordinate axes. The cool thing is that the new scores (call them Z1, Z2, and Z3, if you like) are now all uncorrelated.

Geometrically, the operations are as follows: subtract the mean, rotate the ellipsoid, and then squish or expand the individual axes to make the standard deviations all equal to 1.0. The plot below show three views of the data after standardization. (Don't ask me which axes are L*, a*, and b*, by the way. These are not L*, a*, or b*.)

Standardized version of the L*, a*, and b* variation charts

Not much to look at -- some circular blobs with perhaps a tighter pattern nearer the origin. That's what I would hope to see. 

Here are the stats on this data:

	Mean	Stdev	Skew	Kurtosis
Z1	0.000	1.000	-0.282	-0.064
Z2	0.000	1.000	0.291	0.163
Z3	0.000	1.000	-0.092	-0.658

The mean and standard deviation are exactly 0.000 and 1.000. This is reassuring, but not a surprise. It just means that I did the arithmetic correctly. I designed the technique to do this! Another thing that happened by design is that the correlations between Z1 and Z2, and between Z1 and Z3 are both exactly 0.000. Again, not a surprise. Driving those correlations to zero was the whole point of rotating the ellipsoid, which I don't mind saying was no easy feat.

The skew and kurtosis are more interesting. For an ideal normal distribution, these two values will be zero. Are they close enough to zero? None of these numbers are big enough to raise a red flag. (In the section below entitled "Range for skew and kurtosis", I give some numbers to go by to scale our expectation of skew and kurtosis.)

In the typical doublespeak of a statistician, I can say that there is no evidence that the standardized  color variation is not normal. Of course, that's not to say that the standardized color variation actually is normal, but a statement like that would be asking too much from a statistician. Suffice it to say that it walks like normally distributed data and quacks like normally distributed data.

Dr. Bunsen Honeydew lectures on proper statistical grammar

This is an important finding. At least for this one data set, we know that the standardized scores Z1, Z2, and Z3 can be treated independently as normally distributed variables. Or, as we shall see in the next section, we can combine them into one number that has a known distribution.

Can we expect that all color variation data behaves this nicely when it is standardized by ellipsification? Certainly not. If the data is slowly drifting, the standardization might yield something more like a uniform distribution. If the color is bouncing back and forth between two different colors, then we expect the standardized distributions to be bi-modal. But I intend to look at a lot of color to try to see if 3D normal distribution is the norm for processes that are in control.

In the words of every great research paper every written, "clearly more research is called for".

The Zc statistic

I propose a statistic for SPC of color, which I call Zc. This is a generalization of the Z statistic that we all know and love. This new statistic could be applied to any multi-dimensional data that we like, but I am reserving the name to apply to three-dimensional data, in particular, to color data. (The c stands for "color". If you have trouble remembering that, then note that c is the first letter of my middle name.)

Zc is determined by first ellispifying the data set. The data set is then standardized, and then each data point is reduced to a single number (a scalar), as described in the plot below. The red points are a standardization of the data set we have been working with.the data set we have been working with. I have added circles at Zc of 1, 2, 3, 4. Any data points on one of these circles will have a Zc score of the corresponding circle. Points in between will have intermediate values, which are the distance from the origin. Algebraically, Zc is the sum in quadrature of the individual three components, that is to say, the square root of the sum of the squares of the three individual components.

A two-dimensional view of the Z scores

Now that we have standardized our data into three uncorrelated random variables that are (presumably) Gaussian with zero mean and unit standard deviation, we can build on some established statistics. The sum of the squares of our standardized variable will follow a chi-squared distribution, and the square root of the sums of the squares will follow a chi distribution. Note that this quantity is the distance from the data point to the origin.

Chi is the Greek version of our letter X. It is pronounced with the hard K sound, although I have heard neophytes embarrass themselves by pronouncing it with the ch sound. To make things even more confusing, there is a Hebrew letter chai which is pronounced kinda like hi, only with that rasping thing in the back of your throat. Even more confusing is the fact that the Hebrew chai looks a lot like the Greek letter pi, which is the mathematical symbol for all things circular like pie and cups for chai tea. But the Greek letter chi has nothing to do with either chai tea, or its Spoonerism tai chi.

Whew. Glad I got that outa the way.

Why is it important that we can put a name on the distribution? This gives us a yardstick from which to gauge the probability that any given data point belongs to the set of typical data. The table below gives some probabilities for the Zc distribution. Here is an example that will explain the table a bit. The fifth row of the table says that 97% of the data points that represent typical behavior will have Zc scores of less than 3.0. Thus the chance that a given data point will have a Zc score larger than that is 1 in 34.

Levels of significance of Zc

Zc	P(Zc)	Chance
1.0	0.19875	1
1.5	0.47783	2
2.0	0.73854	4
2.5	0.89994	10
3.0	0.97071	34
3.5	0.99343	152
4.0	0.99887	882
4.5	0.99985	6623
5.0	0.99999	66667

The graph below is a run time chart of the Zc scores for the 204 data points that we have been dealing with. The largest score is about 3.5. We would be hard pressed at calling this an aberrant point, since the table above says that there is a 1 in 152 chance of such data happening at random. By the way, we had close to 152 data points, so we should expect 1 data point above 3.5. A further test: I count eight data points where the Zc score is above 3.0. Based on the table, I expect about 6.

My conclusion is that there is nothing funky about this data.

Runtime chart for Zc of the solid magenta patches

Where do we draw the line between common cause and special cause variation? In traditional SPC, we use Z> 3 as the test for individual points. Note that for a normal distribution, the probability of Z< 3 is 0.99865, or one chance in 741 of Z< 3.0. This is pretty close to the probability of Zc< 4 for a chi distribution. In other words, if you are using Z> 3 as a threshold for QC with normally distributed data, then you should use Zc> 4 when using my proposed Zc statistic for color data. Four is the new three.

Provenance for this data

In 2006, the SNAP committee (Specifications for Newspaper Advertising Production) took on a large project to come to some consensus about what color you get when you mix specific quantities of CMYK ink on newsprint. A total of 102 newspapers printed a test form on its presses. The test form had 928 color patches. All of the test forms were measured by one very busy spectrophotometer. The data was averaged by patch type, and it became known as CGATS TR 002.

Some of the patches were duplicated on the sheet for quality control. In particular all of the solids were duplicated. Thus, in the blog post, I was dealing with 204 measurements of a magenta solid patch from 102 different newspaper printing presses.

Range for skew and kurtosis

How do we decide when a value of skew or kurtosis is indicative of a non-normal distribution? Skew should be 0.0 for normal variation, but can it be 0.01 and still be normal? Or 0.1? Where is the cutoff?

Consider this: the values for skew and kurtosis that we compute from a data set are just estimates of some metaphysical skew and kurtosis. If we asked all the same printers to submit another data set the following day, we would likely have a somewhat different value of all the statistics. If we had the leisure of collecting a Gillian or a Brilliant or even a vermillion measurements, we would have a more accurate estimate of these statistical measures. 

Luckily some math guy figgered out a simple formula that allows us to put a reliability on the estimates of skew and kurtosis that we compute.

Our estimate of skew has a standard deviation of sqrt (6 / N). For N = 204 (as in our case) this works out to 0.171. So, an estimate of skew that is outside of the range from -0.342 to 0.342 is suspect, and outside the range of -0.513 to 0.513 is very suspect.

For kurtosis, the standard deviation of the estimate is sqrt (24/N), which gives us a range of +/- 0.686 for suspicious and +/- 1.029 for very suspicious.

Blue skaters

December 19, 2017, 6:00 am

≫ Next: Why is it called "regression"?

≪ Previous: Statistics of multi-dimensional data, example

$

0

0

A friend of mine, Renzo Shamey, was recently quoted by the New York Times. Well, I would like to think he's a friend of mine. More accurately, I would like you to think he's a friend of mine. I mean, he was quoted in the New York Times! What does that tell you about how great I am?!?!?

The article was about speedskaters, and how there is now a propensity for speedskaters to wear blue uniforms. It makes then faster.

The guy in blue is sooooo much faster than the other guy!

Havard Myklebus, a Norwegian sports scientist, explains the science behind the color choice. Quoting from the NYT article:

“What I’ve said is, our new blue suit is faster than our old red suit,” he [Havard] said with a tight smile, “and I stand by that.”

Here is another quote from the article along the same lines:

“It’s been proven that blue is faster than other colors,” said Dai Dai Ntab, a sprint specialist for the Netherlands.

So. There you have it. Blue is faster. This is born out in the animal kingdom. Umm... maybe not.

Fastest animals on land, in sea, in sky, and on sliderule

My best friend, Renzo, explains the science this way:

... based on my knowledge of dye chemistry, I cannot possibly imagine how dyeing the same fabric with two dyes that have the same properties to different hues would generate differing aerodynamic responses.

A brief, but well-deserved rant

The two answers illustrate the dichotomy of Science. Note the capital S. This indicates that the word should be said in an intense whisper -- with great reverence. On the one hand, Science is a book about everything that we know. We look to Science to explain how and why something works. This is the Science that my long-time buddy Renzo was referring to.

A cherished book from my childhood

Havard, who I'm sure would be a bosom-buddy of mine if I ever met him, is hearkening to the other half of the dichotomy of Science, the half that is more of a verb then a noun. This view of Science is more along the lines of "I poured the stuff in the beaker-thingie. When I stirred it, it blew up and singed off one of my eyebrows. I dunno why, but when I repeated the experiment, my other eyebrow was gone."

Science is both the floor wax that underlays our method of the pursuit of knowledge, and the dessert topping of sweet knowledge that we get from this holy pursuit.

(I sincerely hope that sentence makes it into the Guinness Book of World Records for the most beautiful allusion to an SNL skit to help explain the nature of Science. My Dad would have been proud.)

I mention this Science thing cuz I got a bee in my bonnet. When a person who is into homeopathy, or anti-vaxxing, or astrology is presented with Science, they often respond with "Oh, yeah? Well, Science doesn't know everything!" Perhaps Science-As-A-Noun doesn't have a cure for cancer, can't explain why some sub-atomic particles are cuter than others, and can't tell me why I didn't exercise yesterday, but Science-As-A-Verb provides us with a method that will ultimately answer the first two of those questions. And Science-As-A-Verb has demonstrated that homeopathy is ineffective, vaccines are good, and astrology is bogus.

Enough of my rant. Let's get back to the speed of blue.

Faster than a speeding differential equation because of the blue suit?

Psychochromokinesiology

Here is a quote of Renzo's that did not make it into the NYT article:

Psychologically we are influenced by the colors we wear, in fact I am running a study on this very topic at the moment in North Carolina State and our reactions can be influenced by this also.  It has been shown that reaction responses when people are shown red tends to be faster.

Did I mention that Renzo is my closest (and just about only) friend? I look forward to hearing more about his experiment. I have always been fascinated about the intersection between psychology and color science. Full disclaimer; I am a color scientist, but I am not a psychologist. But, I do have psychology. Just ask my therapist. Or my wife.

Color no doubt effects feelings, and it is only logical that this should apply to sports. After all, Dr. Yogi Berra once said: "Baseball is 90 per cent mental. The other half is physical."

Black and agression

Can you guess which guy is the bad guy?

The earliest study on Psychochromokinesiology that I found was from 1988, The Dark Side of Self- and Social Perception: Black Uniforms and Aggression in Professional Sports. They found that the man in black is more likely to go to the penalty box than athletes wearing other colors.

An analysis of the penalty records of the National Football League and the National Hockey League indicate that teams with black uniforms in both sports ranked near the top of their leagues in penalties throughout the period of study.

But, cause or effect? Did they receive more penalties because wearing black makes an athlete more aggressive? Or is this a case of the don't-drive-a-red-car-cuz-the-cops-are-more-likely-to-pull-you-over syndrome? The researchers set up experiments to test both explanations. It turned out that both were true.

Red and performance

Danger, Gene!

But wearing red might be a good thing, perhaps because of the effect on the other team. Red means danger, right? Here is a quote from one study, Psychology: Red enhances human performance in contests, published in Nature:

...across a range of sports, we find that wearing red is consistently associated with a higher probability of winning.

Here is another really technical sounding paper, Red shirt colour is associated with long-term team success in English football, that gives a shout out to red:


A matched-pairs analysis of red and non-red wearing teams in eight English cities shows significantly better performance of red teams over a 55-year period.

Two out of two technical papers choose red uniforms. But why would it matter?

Color's effect on the perception of others

The kids with the red uniforms always got picked first for dodge ball

Another study tried to figger out what went on in the mind of a goalie: Soccer penalty takers' uniform colour and pre-penalty kick gaze affect the impressions formed of them by opposing goalkeepers. They showed goalies video clips of soccer players taking penalty shots, and then asked the goalies for their opinions. The conclusion was that a penalty kicker was perceived as being more competent if they were wearing red than if they were wearing white.

Here is study that suggests that dominance of athletes in red uniforms might be due to bias in judging: When the Referee Sees Red.... In this study, the researchers created two versions of the 11 video clips from a tae kwon do match. The two versions were identical except that the color of the protective gear was switched. In one video, it was red versus blue, and in the other, it was blue versus red. You can watch one of the clips here. They sat 42 experienced referees down in front of the videos and asked them to count points for each athlete. Their results?

...competitors dressed in red are awarded more points than competitors dressed in blue, even when their performance is identical.

Summary

Black is meaner than other colors, and red wins more often than blue. Why is this? There is some evidence that a player changes his or her behavior because of the color they wear. There also is evidence that players react differently because of the colors that other players wear. And, there is also evidence that referees judge players differently based on the color of the uniform.

But I did not find any studies on why a blue uniform would make a skater faster. In the spirit of all research papers written by researchers looking for continued funding, let me say that more research is clearly necessary.

Why is it called "regression"?

December 27, 2017, 7:00 am

≫ Next: Statistical process control of color, a method that works

≪ Previous: Blue skaters

$

0

0

Regression. Such a strange name to be applied to our good friend, the method of least-squares curve fitting. How did that happen?

My dictionary says that regression is the act of falling back to an earlier state. In psychiatry, regression refers to a defense mechanism where you regress – fall back – to a younger age to avoid dealing with the problems that us adults have to deal with. Boy, can I relate to that!

All statisticians recognize the need for regression

Then there’s regression therapy, and regression testing…

Changing the subject radically, the “method of least squares” is used to find the line or curve that "best" goes through a set of points. You look at the deviations from a curve – each of the individual errors in fitting the curve to the points. Each of these deviations is squared and then they are all added up. The least squares part comes in because you adjust the curve so as to minimize this sum. When you find the parameters of the curve that give you the smallest sum, you have the least squares fit of the curve to your data.

For some silly reason, the method of least squares is also known as regression. It is perhaps an interesting story. I have been in negotiations with Random House on a picture book version of this for pre-schoolers, but I will give a preview here.

Prelude to regression

Let’s scroll back to the year 1766. Johann Titius has just published a book that gave a fairly simple formula that approximated the distances from the Sun to all the planets. Titius had discovered that if you subtract a constant from the size of the each orbit, the planets all fell in a geometric progression. After subtracting a constant, each planet was twice as far from the Sun as the one previous. Since Titius discovered this formula, it became known as Bode’s law.

I digress in this blog about regressing. Stigler’s law of eponymy says that all scientific discoveries are named after someone other than the original discoverer. Johann Titius stated his law in 1766. Johann Bode repeated the rule in 1772, and in a later edition, attributed it to Titius. Thus, it is commonly known as Bode’s law. Every once in a while it is called as the Titius-Bode law.

The law held true for six planets: Mercury, Venus, Earth, Mars, Jupiter, and Saturn. This was interesting, but didn’t raise many eyebrows. But when Uranus was discovered in 1781, and it fit the law, people were starting to think seriously about Bode’s law. It was more than a curiosity; it was starting to look like a fact.

But there was just one thing I left out about Bode’s law – the gap between Mars and Jupiter. Bode’s law worked fabulous if you pretended there was a mysterious planet between these two. Mars is planet four and we will pretend that Jupiter is planet six. Does planet five exist?

Now where did I put that fifth planet???

Scroll ahead to 1800. Twenty four of the world’s finest astronomers were recruited to go find the elusive fifth planet. On New Year’s Day of 1801, the first day of the new century, a fellow by the name of Giuseppe Piazzi discovered Ceres. Since it was moving with respect to the background of stars, he knew it was not a star, but rather something that resided in our the solar system. At first Piazzi thought it was a comet, but he also realized that it could be the much sought after fifth planet.

How could he decide? He needed to have enough observations over a long enough time period of time so that the orbital parameters of Ceres could be determined. Piazza observed Ceres a total of 24 times between January 1 and February 11. Then he fell ill, suspending his observations. Now, bear in mind that divining an orbit is a tricky business. This is a rather short period of time from which to determine the orbit.

It was not until September of 1801 that word got out about this potential planet. Unfortunately, Ceres had slipped behind the Sun by then, so other astronomers could not track it. The best guess at the time was that it should again be visible by the end of the year, but it was hard to predict just where the little bugger might show his face again.

Invention of least squares curve fitting

Enter Karl Friedrich Gauss. Many folks who work with statistics will recall his name in association with the Gaussian distribution (also known as the normal curve and the bell curve). People who are keen on linear algebra will no doubt recall the algorithm called “Gaussian elimination”, which is use to solve systems of linear equations. Physicists are not doubt aware of the unit of measurement of the strength of a magnetic field that was named after Gauss. Wikipedia currently lists 54 things that were named after Gauss.

More digressing...As is the case of every mathematical discovery, the Gaussian distributions was named after the wrong person.The curve was discovered by De Moivre. Did I mention Stigler? Oh... while I am at it, I should mention that Gaussian elimination was developed in China when young Gauss was only -1,600 years old.. Isaac Newton independently developed the idea about 1670. Gauss improved the notation in 1810, and thus the algorithm was named after him.

Back to the story. Gauss had developed the idea of least squares in 1795, but did not publish it at the time. He immediately saw that the Ceres problem was an application for this tool. He used least squares to fit a curve to the existing data in order to ascertain the parameters of the orbit. Then he used those parameters to predict where Ceres would be when it popped its head out from behind the Sun. Sure enough, on New Year’s eve of 1801, Ceres was found pretty darn close to where Gauss had said it would be. I remember hearing a lot of champagne corks popping at the Gaussian household that night! Truth be told, I don't recall much else!

From Gauss' 1809 paper "Theory of the Combination of Observations Least Subject to Error"

The story of Ceres had a happy ending, but the story of least squares got a bit icky. Gauss did not publish his method of least squares until 1809. This was four years after Adrien Marie Legendre’s introduction of this same method. When Legendre found out about Gauss’ claim of priority on Twitter, he unfriended him on FaceBook. It's sad to see legendary historical figures fight, but I don't really blame him.

In the next ten years, the incredibly useful technique of regression became a standard tool in many scientific studies - enough so that it became a topic in text books.

Regression

So, that’s where the method of least squares came from. But why do we call it regression?

I’m going to sound (for the moment) like I am changing the subject. I’m not really, so bear with me. It’s not like that one other blog post where I started talking about something completely irrelevant. My shrink says I need to work on staying focused. His socks usually don't match.

Let’s just say that there is a couple, call them Norm and Cheryl (not their real names). Let’s just say that Norm is a pretty tall guy, say, 6’ 5” (not his real height). Let’s say that Cheryl is also pretty tall, say, 6’ 2” (again, not her real height). How tall do we expect their kids to be?

I think most people would say that the kids are likely to be a bit taller than the parents, since both parents are tall – they get a double helping of whatever genes there are that make people tall, right?

One would think the kids would be taller, but statistics show this is generally not the case. Sir Francis Galton discovered this around 1877 and called it “regression to the mean”. Offspring of parents with extreme characteristics will tend to regress (move back) toward the average.

Why would this happen?

As with most all biometrics (biological measurements), there are two components that drive a person’s height – nature and nurture, genetics and environment. I apologize in advance to the mathaphobes who read this blog, but I am going to put this in equation form.

Actual Height = Genetic height + Some random stuff

Here comes the key point: If someone is above average in height, then it is likely that the contribution of “some random stuff” is a bit more than average. It doesn’t have to be, of course. Someone can still be really tall and still shorter than genetics would generally dictate. But, if someone is really tall, it’s likely that they got two scoops: genetics and random stuff.

So, what about the offspring of really tall people? If both parents are really tall, then you would expect the genetic height of the offspring to be about the same as that of the parents, or maybe a bit taller. But (here comes the second part of the key point) if both parents were dealt a good hand of random stuff, and the hand of random stuff that the children are dealt is average, then it is likely that the offspring will not get as good a hand as the parents. 

The end result is that the height of the children is a balance between the upward push of genetics and the downward push of random stuff. In the long run, the random stuff has a slight edge. We find that the children of particularly tall parents will regress to the mean.

We expect the little shaver to grow up to be a bit shorter than mom and pop

Galton and the idea of "regression towards mediocrity"
Francis Galton noticed this regression to the mean when he was investigating the heritability of traits. He started doing all kinds of graphs and plots and stuff, and chasing his slide rule after bunches of stuff. He had graphs like the one below, showing the distribution of the heights of adult offspring as a function of the mean height of their parents.

From Galton's paper "Regression towards mediocrity in hereditary stature", 1886

In case you're wondering, this is what we would call a two-dimensional histogram. Galton's chart above is a summary of 930 people and their parents. You may have to zoom in to see this, but there are a whole bunch of numbers arranged in seven rows and ten columns. The rows indicate the average height of the parent, and the columns are the height of the child. Galton laid these numbers out on a sheet of paper (like cells in a spreadsheet) and had the clever idea of drawing a curve that traced through cells with similar values. He called these curves isograms, but the name didn't stick. Today, they might be called contour lines; on a topographic plot, they are called isoclines, and on weather maps, we find isobars and isotherms.   

Galton noted that the isograms on his plot of heights were a set of concentric ellipses, one of which is shown in the plot above. The ellipses were all tilted upward on the right side.

As an aside, Galton's isograms were the first instance of ellipsification that I have seen. Coincidentally, the last blog post that I wrote was on the use of ellipsification for SPC of color data. I was not aware of Galton's ellipsification when I started writing this blog post. Another example of the fundamental inter-connectedness of  all things. Or an example of people finding patterns in everything!

Galton did not give a name to the major axis of the ellipse. He did speak about the "mean regression in stature of a population", which is the tilt of the major axis of the ellipse. From this analysis, he determined that number to be 2/3, which is to say, if the parents are three inches taller than average, then we can expect (on average) that the children be two inches above average.

So, Galton introduced the word regression into the field of statistics of two variables. He never used it to describe a technique for fitting a line to a set of data points. In fact, the math he used to derive his mean regression in stature bears no similarity to the linear regression by least squares that is taught in stats class. Apparently, he was unaware of the method of least squares.

Enter George Udny Yule
George Udny Yule was the first person to misappropriate the word regression to mean something not related to "returning to an earlier state". In 1897, he published a paper called On the Theory of Correlation in the Journal of the Royal Statistical Society. In this paper, he borrowed the concepts implied by the drawings from Galton's 1886 paper, and seized upon the word regression. In his own words, "[data points] range themselves more or less closely round a smooth curve, which we shall name the curve of regression of x on y." In a footnote, he mentions the paper by Galton and the meaning that Galton had originally assigned to the word.

In the rest of the paper, Galton lays out the equations for performing a least squares fit. He does not claim authorship of this idea. He references a textbook entitled Method of Least Squares (Mansfield Merriman, 1894). Merriman's book was very influential in the hard sciences, having been first published in 1877, with the eighth version in 1910.

So Yule is the guy who is responsible for bringing Gauss' method of least squares into the social sciences, and in calling it by the wrong name.

Yule reiterates his word choice in the book Introduction to the Theory of Statistics, first published in 1910, with the 14th edition published in 1965. He says: In general, however, the idea of "stepping back" or "regression" towards a more or less stationary mean is quite inapplicable ... the term "coefficient of regression" should be regarded simply as a convenient name for the coefficients b1 and b2.

So. There's the answer. Yule is the guy who gave the word regression a completely different meaning. How did his word, regression, become so commonplace, when "least squares" was a perfectly apt word that had already established itself in the hard sciences? I can't know for sure.

The word regression is a popular word on my bookshelf

Addendum

Galton is to be appreciated for his development of the concept of correlation, but before we applaud him for his virtue, we need to understand why he spent much of his life measuring various attributes of people, and inventing the science of statistics to make sense of those measurements.

Galton was a second cousin of Charles Darwin, and was taken with the idea of evolution. Regression wasn't the only word he invented. He also coined the word eugenics, and defines it thus:

"We greatly want a brief word to express the science of improving stock, which is by no means confined to questions of judicious mating, but which, especially in the case of man, takes cognisance of all influences that tend in however remote a degree to give to the more suitable races or strains of blood a better chance of prevailing speedily over the less suitable than they otherwise would have had. The word eugenics would sufficiently express the idea..."

Francis Galton, Inquiries into Human Faculty and its Development, 1883, page 17

The book can be summarized as a passionate plea for the need of more research to identify and quantify those traits in humans that are good versus those which are bad. But what should be done about traits that are deemed bad? Here is what he says:

"There exists a sentiment, for the most part quite unreasonable, against the gradual extinction of an inferior race. It rests on some confusion between the race and the individual, as if the destruction of a race was equivalent to the destruction of a large number of men. It is nothing of the kind when the process of extinction works silently and slowly through the earlier marriage of members of the superior race, through their greater vitality under equal stress, through their better chances of getting a livelihood, or through their prepotency in mixed marriages."

Ibid, pps 200 - 201

It seems that Galton favors a kindler, gentler form of ethnic cleansing. I sincerely hope that all my readers are as disgusted by these words as I am.

Statistical process control of color, a method that works

January 16, 2018, 4:01 pm

≫ Next: Munsell - the Father of Color Science? (part 1)

≪ Previous: Why is it called "regression"?

$

0

0

I have preached quite a bit about the shortcomings of many methods for statistical process control of color data. I reference all the previous blogs at the end of this blog post, just in case you missed them. My take on the topic can be summarized by an anecdote about President Coolidge. After attending a church service, he was asked about the topic of the sermon. "Sin." The followup question was to ask what the minister had to say about sin. "He was against it."

It's time to unveil my technique.

Looking at magenta

Magenta is an interesting ink. It is well known that as you increase the ink film thickness (or pigment concentration), it will change in hue toward red. I know for a fact that this is well-known, since I blogged about the hue shift of inks before. After all, anyone who is anyone reads my blog. 

(Magenta is not such an odd duck, when it comes to colorants. Cyan ink is another printing ink that has this hook. I know from the physics of color that there are a lot of colorants in other industries that will do this same thing.)

The red ellipse in the diagram shows the area of the magenta ink trajectory where magenta is normally run. There are two aspects of this plot that might be disconcerting.

First, the ink trajectory is curved. One might therefor expect that the normal variation in color might wind up being shaped like a kidney bean, rather than a nice ellipsoidal jelly bean. Or maybe... I shudder to think of it... like a cashew! 

Why is this a concern? The idea of SPC is based on being able to tell the difference between typical  and atypical behavior. Ellipsoids are kinda easy to describe mathematically, so it is kinda easy to tell what is inside the ellipsoid and what is outside. I searched through my copy of the NIST Handbook of Mathematical Functions. The word cashew does not appear.

Second, it is clear that, even if the scatter of points for magenta in L*a*b* can be suitably approximated by an ellipsoid, the ellipsoid is not tilted toward the origin, as was the example of yellow in from the previous blog post on this topic.

Whether or not the first issue is a true matter of concern depends upon the amount of curvature present over the range of typical variation, and also the magnitude of other contributors to the variation. I will provide one example of the variation of magenta ink where the hook is not a problem. We shall see in this analysis that the odd direction of the tilt of magenta doesn't make a bit of a difference.

Magenta variation in newspaper data

I spoke before about a data set of test targets printed by 102 newspaper printers. One hundred printers, each printing 928 patches, with (in particular) two magenta solid patches on each one. The image below shows the variation in a*b* of all the solid magentas. 

You will note that it is kinda elliptical, with a decided tilt that is not toward neutral gray. Kinda what we would expect. Just visually, I can't see any sign of kidney-beaning. This, despite the fact that the variation is quite large, something like 10 ΔE from the two extremes. Then again, the other variations may just be doing a great job of hiding any curvature that is present. At any rate, I am going to make the bold assertion that we are not going to be making any beans and rice with this data set.

Below is 3D view of that same data. The height axis of the plot is L*, from 40 to 70. The (mostly) right to left axis is a*, from 30 to 60. The front to back axis is b* from -15 to 15. 

Note that the ellipsoid is not only tilted kinda away from neutral gray, but it is also tilted downward, which makes sense, I guess. As you make the ink richer in color (more pigment) you also make it darker.

A note about the image above. You should see an animation, with the set of points rotating around. If the animation isn't working, try a different browser, or try downloading the image and displaying it with some other app.  I found that Microsoft Office Manager and Paint did not display the animation. Microsoft Photos, Internet Explorer, and Windows Media Player do.

Once again, I don't have much of a problem saying that this data resembles an ellipsoid. At any rate, it looks a lot more like an ellipsoid than a box, which is the assumption that is made if we separately analyze ΔL*, Δa*, and  Δb*.

Previous blog posts

Here is the first post in a series of four blog posts about the futility of using ΔE for statistical process control. The first post ends with a link to the second in the series, and so on. 

I wrote a more recent blog post that looked at a common process management tool, the cumulative relative frequency plot (CRF) with ΔE data. Again, I gave some warnings about trying to make much out of the shape of the curve.

Then I wrote a post about the practice of applying SPC on ΔL*, Δa*, and  Δb*. My conclusion is that this is better in some cases, but there are some very reasonable distributions of color data where the method falls apart.

Munsell - the Father of Color Science? (part 1)

February 7, 2018, 6:00 am

≫ Next: Munsell - the Father of Color Science? (part 2)

≪ Previous: Statistical process control of color, a method that works

$

0

0

Albert Munsell. <pause for dramatic effect> Should he be credited with being the Father of Color Science? <another pause> That's my topic for today's blog post.

The question should come as no big surprise to anyone who has been following my blog. So far, his name has shown up in 20 of my blog posts. My wife is even starting to get a bit suspicious. This coming June, he will have been gone for 100 years, but jealousy can make folks a little crazy.

By the way... The 100th anniversary of the founding of the Munsell company will be celebrated this summer. Munsell Centennial Symposium will be held June 10 - 15, 2018 in his old stomping grounds, Boston. Your's truly will be keynoting, and (bonus points) I will be giving a tutorial on color measurement devices.

Let's look at some of Munsell's legacy to decide whether he deserves the honor. Today we look at Munsell's contribution to color science education.

Munsell was an enthusiastic teacher of color

Munsell's first claim on the coveted title of Father of Color Science was that he had a passion for teaching about color. Let me give some supporting evidence.

Albert Munsell, the Father of the Color Science Kids, Margaret and A.E.O.

He filed for a patent in 1899 (US Patent 640,972, A Color Sphere and Mount) for a color sphere, which was a globe, with the rainbow colors painted around the equator, with gradations of those colors mixed with white heading up to the North Pole, and with similar gradations mixed with black in the Southern Hemisphere. Quoting from his patent: "The object of my invention is to provide a spherical color chart for educational purposes." In addition, Munsell filed for a patent for a Spinning Top in 1902. One could affix colored cards to the spinning surface of the top "for the purpose of producing novel color effects". It sure sounds like Munsell was in the business of color edu-tainment to me!

Munsell's Color Sphere (left) and top (right)

He worked toward standardizing the teaching of color. In 1904, Munsell started working with teachers in Boston on a primer for teaching color in grades 4 through 9. Munsell developed a set of 22 crayons in 1906. This line was eventually added to the set of Crayola crayons sold by Binney and Smith. In 1917, the Munsell Color Company was formed to sell art supplies to schools.

Who doesn't remember the smell of a fresh box of these crayons at the start of the school year?

Incidentally, the word crayon dates back to the mid 1600's. It was Alice Binney, the wife of founder Edwin Binney who coined the word crayola, a conjunction of the prefix cray- from crayon and -ola, which means oleaginous (oily). This suffix was popular for products in the day: Mazola, granola, Victrola, and of course, Shinola. Now that you read that, don't let anyone tell you that you don't know cray from crayola!  

So, does this qualify Albert Munsell for the title of Father of Color Science? While his work was impressive, sadly, I don't think he deserves the title for these efforts.

Not to malign the guy, but Albert was not the only evangelist for proper color education at the turn of the last century. A gentleman by the name of Milton Bradley was another early chromo-vangelist. Yes. That Milton Bradley. The guy who invented the Game of Life, Operation, Battleship, and  of course Candyland. Ohhh... the late night Candyland parties we used to have when I was in college!

Bradley's colored paper samples

In Bradley's book Elementary Color he describes the Bradley System of Color Instruction, which aims "to offer a definite scheme and suitable material for a logical presentation of the truths regarding color in nature and art to the children of primary schools." The third edition of this book was published in 1895, a few years prior to the start of Munsell's colorful evangelical career. 

By the way, I should mention that Milton Bradley filed for an patent for a Color Disk Rotating Mechanism in 1893, and for a Color Mixing Top seven years before Munsell's filing for a color mixing top. To add insult to injury, Bradley developed a line of crayons in 1895, and had a business relationship with Binney and Smith for a few years, starting in 1905.

My ruling so far is that Albert Munsell is, at best, one of two Godfathers of Color Science Education. Stay tuned for the next blog post, where I investigate whether Munsell invented the three-dimensional color space!

Would you like to hear me rehash this topic in the same dreary and boring manner, but with the benefit of my dull and boring voice? Live? With the opportunity to heckle me with questions??? Sign up for the ISCC webinar.

Munsell - the Father of Color Science? (part 2)

February 12, 2018, 6:00 am

≫ Next: Is my color process going awry?

≪ Previous: Munsell - the Father of Color Science? (part 1)

$

0

0

Albert Munsell has been called the Father of Color Science. In the previous blog post, I looked at whether he earned that accolade through his crusade to put the Science into Color Science Education. I concluded that he would probably have to share this with Milton Bradley -- the board game magnate. I dunno, though. Saying that Munsell and Bradley are both of the Fathers of Color Science might get a bit weird for some.

Before I continue, Munsell is held in esteem by real color scientists, not just color science wannabes who write corny blogs on color in hopes of being invited to the real people parties. One of those cool people parties is the Munsell Centennial Color Symposium, June 10-15, 2018, MassArt, Boston, MA.

So far, I have only gotten as far as being invited to give a webinar for cool people, which is based on this series of posts.  If you are reading this before Feb 21, 2018, then there is still time to sign up. As further.

Today, I try another possible explanation for why Munsell might be due the honor. Albert Munsell developed the Munsell Color System. Unlike previous two-dimensional color systems, the Munsell Color Space is three-dimensional! That made it way cooler. All of the avid readers of my blog even know why color is three-dimensional.

World famous Color Science Model admires the Munsell Color System

But was he the first to bring 3D to color?

Munsell invented the idea of a three-dimensional color space?

Here is a quote from the Introduction of Munsell's book A Color Notation System (1919). (The Introduction was written by H. E. Clifford. Evidently Clifford was his publicist. The world famous Color Science Model shown above is my publicist.)

"The attempt to express color relations by using merely two dimensions, or two definite characteristics, can never lead to a successful system. For this reason alone the system proposed by Mr. Munsell, with its three dimensions of hue, value, and chroma, is a decided step in advance over any previous proposition."

Hue, value, and chroma

That kinda sounds like three dimensional color was Munsell's idea?

Here is another piece of evidence suggesting that Munsell may have been the guy that brought 3D color to a cinema near you. US Patent #824,374 for a Color Chart or Scale was issued to Munsell in 1906. His disclosure states: "It may assist in understanding the order of arrangement of my charts to know that the idea was suggested by the form of a spherical solid subdivided through the equator and in parallel planes thereto, ..."

Doncha just love drawings from old patents?

Fig. 2 above shows a page where the hues of the rainbow are arranged around the perimeter, with them all fading to gray at the center. This is but one page of color. Previous pages would have a brighter version of this, and subsequent pages would be darker. Fig. 1 shows a cut-away version of these pages assembled into a book.

So, he got the patent! Case closed. Munsell deserves to be the Father of Color Science.

Or did he patent the color space?

But... hold on a sec. Another part of the disclosure in the patent refers to "the three well-known constants or qualities of color -- namely, hue, value or luminosity, and purity of chroma..." In the patent biz, we would refer to that hyphenated word well-known as a pretty clear admission of prior art!

Clearly Munsell did not invent the idea of using three coordinates to identify unique colors. This is why I keep telling my dogs that you have to read patents very carefully to understand what is being patented. My cute little puppies are always ready to get out the pitchforks and torches after doing a quick read of a patent.

In Munsell's paper A Pigment Color System and Notation (The Journal of Psychology, 1912), he refers to a number of previous color ordering schemes by "Lambert, Runge, Chevreul, Benson, and others".

A slice of Munsell

So, I did a little investigation. Munsell also mentioned Ogden Rood as an experimenter in color. I dug out a book named Modern Chromatics, by Ogden Rood. I should point out that using the word modern in the title of a book may not be such a good idea if you want the book to be around for a while. This book was published a while ago, like thoroughly before Modern Millie, like in 1879.

The diagrams below are from Rood's book. They look kinda like representations of three-dimensional things to me!

Cross section of Rood's color cylinder and color cone

Not only does Rood's book predate the Munsell patent by about 30 years, but on page 215, he pushed the discovery of three dimensional color back by a full century: "This colour-cone is analogous to the color pyramid described by Lambert in 1772." That was soooo rood of him!

(That was probably the worst pun of my life. I apologize to the anyone whose sense of humor was offended.)

How about these other color systems?

I stumbled on a website called colorsystem.com which chronicles more color systems that you can shake a crayon at. Here is their list of the three-dimensional color systems which predate Munsell. Are you ready?

Johann Heinrich Lambert, 1772

Tobias Mayer, 1775 

Philipp Otto Runge's Farbenkugel (1810)

I just love the name of his color space. In addition to being a world famous Color Scientist Model, my wife makes a pretty decent savory kugel.

Michel Eugene Chevreul, 1839

William Benson, 1868

Benson touts this as both an additive color space and a subtractive one. Orient it one way and you get RGB axes. Orient it another, and you get (what I would call) CMY. He called them yellow, sea-green, and pink. I have used this trick in classes for years. I had no idea that it was invented so long ago.

Wilhelm von Bezold, 1874

Wilhelm Wundt, 1874

So, including Rood's, we have eight different suggestions for a three-dimensional color space, all of which came before Munsell. Oh... wait, I almost forget the earliest one.

Robert Grosseteste, 1230

This gentleman deserves a bit of comment. The colorsystem entry on Grosseteste is a bit sparse, if you ask me. First, Grosseteste has to share a webpage with Leon Battista Alberti and Leonardo da Vinci. I would be honored to share a webpage with da Vinci, but colorsystem didn't mention that Grosseteste's color system was likely the first three-dimensional color system ever conceived.

I do not mean to malign the good folks at colorsystem (although that would be pretty much in line with my reaction to anyone who knows more than I do). I love their website. I think the whole cover-up of Grosseteste's three-dimensional color system was part of a bigger conspiracy to deprive him of his rightful place in the History of Science. In the words of David Knowles (in The Evolution of Medieval Thought, p. 281, "[Grosseteste] is now only a name ... because his chief work was done in fields where he could light a torch and hand it on, but could not himself be a burning flame for ever."

Roger Bacon, who was one of the thinkers that led our way into the renaissance, would become one of the burning flames kindled by Grosseteste. Thus, we see that Robert Grosseteste had two degrees of separation from Kevin Bacon, who was in the movie Apollo 13, which kinda had something to with with science.

Here is a quote from an in-depth study by some people who sound gosh-darn scholarly. The quote is pertinent to the debate over the first three-dimensional color space: "De colore [the paper from Grosseteste] dates from the early thirteenth century and contains a convincing argument for a three-dimensional colour space that does not follow the linear arguments that Grosseteste had inherited from previous philosophers..." 

Back to the Munsell Color Space

It would appear that my original premise was far from being correct. Munsell did not create the first three-dimensional color space.

BUT!!!! The astute picture looker will notice something critical. Rood gave us color spaces that were a cylinder and a cone. Bezold also gave us a cone, and Grosseteste gave us a double cone. Lambert's was a pyramid. Mayer's was a triangular prism. Runge, Chevreul, and Wundt all provided spheres. The Benson color space is a cube.

Please do me the favor of scrolling up to the diagram entitled "A slice of Munsell". Please do me the favor of identifying the shape of that slice. This reminds me of the time when my shrink gave me a Rorschach test. Him: "What does this ink blot look like?" Me: "An ink blot." I failed the test.

Most of the drawings in Munsell's A Color Notation System depict his color space as being a sphere, but there are a few drawings like Fig, 20 (above) that show that his color space is irregular. In his own words, "Fig. 20 is a horizontal chart of all the colors which present middle value (5), and describes by an uneven contour the chroma of every hue at this level."

The last pages of this book are color plates that are slices from his Color Atlas. Note the distinct non-standard-shapedness of this.

Why was Munsell's color space groundbreaking?

We finally come to the unique and revolutionary feature: The Munsell Color Space is not a standard geometric shape. As shown below, the high chroma red hues stick out a lot further than the blue ones. It's hard to see this, but the yellow hues with the highest chroma are near the top, whereas the richest purples are nearer the bottom.

The Munsell color solid

Munsell took the non-intuitive road not taken, and that has made all the difference. That will be taken up in the next exciting installment of this series! 

---

Is my color process going awry?

February 14, 2018, 6:00 am

≫ Next: Munsell - the Father of Color Science? (part 3)

≪ Previous: Munsell - the Father of Color Science? (part 2)

$

0

0

This blog is a first in a series of blog posts giving some concrete examples of how the newly-invented technique of ColorSPC and ellipsification can be used to answer real-world questions being asked by real-world people about real-world problems for color manufacturers.

So, picture this scenario. I am running a machine that puts color onto (or into) a product. Maybe it's some kind of printing press; maybe it mixes pigment into plastic; maybe this is about dyeing textiles or maybe it's about painting cars. The same principles apply.

John the Math Guy really Lays color SPC on the line

Today's question: I got this fancy-pants spectrophotometer that spits out color measurements of my product. How can I use it to alert me when the color is starting to wander outside of its normal operating zone?

An important distinction

There are two main reasons to measure parts coming off an assembly line:

     1. Is the product meeting customer tolerances?

     2. Is my machine behaving normally?

Conformance and SPC (statistical process control). These are intertwined. Generally, one implies the other. But consider two scenarios where the two answers are different. 

It could be that the product is meeting tolerances, but the machine is a bit wonky. Not wonky enough to be spitting out red parts instead of green, but there is definitely something different than yesterday. Should we do anything about this? Maybe, maybe not. It's certainly not a reason to run out of the building with our hair on fire. But it could be your machine's way of asking for a little TLC in the form of preventative maintenance.

Or it could be that your machine is operating within its normal range, and is producing product that is outside the customer tolerances. This the case you need to worry about. Futzing with the usual control knobs ain't gonna bring things in line. You need to change something about your process.

Use of DE for SPC

The color difference formulas, such as DE₀₀, were designed specifically to be industrial tolerances for color. While DE₀₀ may well be the second ugliest formula ever developed by a sentient being in this universe, it does a fair job of correlating with our own perception of whether two colors are an acceptable match. 

But is it a good way to assess whether the machine is operating in a stable manner? I mean, you just track DE over time, and if it blips, you know something is going on. Right? Let's try it out on a set of real data.

The plot below is a runtime chart of just over 1,000 measurements of pink spot color that I received from Company B. These are all measurements from a single run. I don't know for sure what the customer tolerance was, but I took a guess at 3.0 DE₀₀, and added that as an orange dashed line.

It sure looks like a lot of measurements were out of tolerance!

Uh-oh. It looks like we got a problem. There are a whole lot of measurements that are well above that tolerance... maybe one out of three are out of tolerance?

But maybe it's not as bad as it looks. The determination lies in how one interprets tolerance. Here is one interpretation from a technical report from the Committee for Graphic Arts Technologies Standards (CGATS TR 016, Graphic technology — Printing Tolerance and Conformity Assessment):

"The printing run should be sampled randomly over the length of the run and a minimum of 20 samples collected. The metric for production variation is the 70th percentile of the distribution of the color difference between production samples and the substrate-corrected process control aims."

TR 016 defines a number of conformance levels. It says that 3.0 DE₀₀  is "Level II conformance", so the orange dashed line is a quite reasonable acceptance criteria for a press run. But a runtime chart is not at all useful for identifying those "Danger Will Robinson" moments. I mean, how do you decide if a single measurement is outside of a tolerance that requires 20 measurements? 

If we want to do SPC, then we must set the upper control limit differently.

Use of DE for SPC, take 2

The basic approach from statistical process control -- the whole six sigma shtick -- is to set the upper control limit based on what the data tells us about the process, and not based on customer tolerances. It is traditional to use the average plus three times the standard deviation as the upper limit. For our test data set, this works out to 5.28 DE₀₀.

The process looks in control now!

This new chart looks a lot more like a chart that we can use to identify goobers. In fact, I did just that with the two red arrows. Gosh darn it, everything looks pretty good.

But I think we need a bit closer look at what the upper limit DE means. The following pair of plots give us a perspective of this data in CIELAB. The plot on the left is looking down from the top at the a*b* values. The plot on the right is looking at the data points from the side with chroma on the horizontal axis and L* on the vertical.

The green dots are each of the measurements. The red diamond is the target color, and the ovoids are the upper limit tolerances of 5.28 DE₀₀. (Note: in DE₀₀, the tolerance regions are not truly ellipses, but are properly called ovoids. One should ovoid calling them ellipses, and also ovoid making really bad puns.)

Those are some big eggs!

The next image is  closeup of the C*L* plot, showing (with red arrows) the small set of wonky points that were identified with the DE runtime chart. I would say that these are pretty likely to be outliers. But look at the smattering of points that are well outside the cluster of data points, but are still within the ovoid that serves as the upper limit for DE. These should have stuck out in the runtime chart, if it were doing its job), but are deemed OK.

Wonkyville

Now, listen carefully... If you are using a runtime plot of "DE₀₀ from the target color", you are in effect saying that everything within the ovoids represents normal behavior for your process. So long as measurements are within those ovoids, you will conclude that nothing has changed in your process. That's just silly talk!

Here is my summary of DE runtime charts: JUST SAY NO! Well... unless your are looking at conformance, and your customer tolerance is an absolute, as in, "don't you never go above 4 DE₀₀!"

Use of Zc for a SPC

I know this was a long time ago, but remember the Z statistic from Stats 101? You compute the average and standard deviation of your data, and then normalize your data points to give you a parameter called Z. If a data point had a Z value that was much smaller than -3, or much larger than +3, then it was suspicious. This is mathematically equivalent to what's going on with the upper limit in a runtime chart.

I have extended this idea to three-dimensional data (such as color data). I call the statistic Zc. This is the keystone of ColorSPC.

Now, remember back when I showed the CIELAB plots of the data along with a DE₀₀ ovoid? Didn't you just want to grab a red pencil and draw in some ellipses that represented the data better? That's what I did, only I used my slide rule instead of a pencil. There is a mathematical algorithm that I call ellipsification that adjusts the axes lengths and orientation of a three-dimensional ellipsoid to "fit" the data. Ellipsification is the keystone of ColorSPC.

Ellipsification charts in CIELAB

The concentric ellipses in the drawings above are the points where Zc = 1, 2, 3, and 4. That is to say, all points on the innermost ellipse have Zc of 1. All points between the innermost and the next ellipse have Zc between 1 and 2.

Zc is a much better way to do SPC on color data. Here is a runtime plot of Zc for this production run. The red dashed line is set to 3.75. That number is the 3D equivalent of the Z = 3 upper limit used in traditional SPC.

Finally, a runtime chart we can believe!

As can easily be seen (if you click on the image, and then get out a magnifying glass) this view of the data provides us with a much better indication of data points which are outside of the typical variation of the process. Nine outliers are identified, and many of them stick out like sore thumbs. Kinda what we would expect from the CIELAB plots.

But wait!

In the previous DE analysis, we computed DE from the target value. In a paper by Brian Gamm (The Analysis Of Inline Color Measurements For Package And Labels Printing Using Statistical Process Monitoring Techniques, TAGA 2017), he pointed out this problem with DE runtinme charts, and advocated the use of the DE, but with DE measured from the average L*a*b* value, rather than the target. The graphs below show the result of this analysis on our favorite data set.

DE₀₀ ovoids based on computing color difference from average

There is no doubt that this is much better for the purposes of SPC. The ovoids are considerably tighter to the data than when the target DE₀₀ was used. So, this approach will disregard far fewer actual outliers.

It is interesting to note that the DE₀₀ ovoid in L*a*b* is similar to the to the ovoid produced by ellipsifcation. Larger, and not quite as eccentric, but similar in orientation. This is a good thing, and will often be the case. This will not be the case for any pigments that have a hook, which is to say, those that change in hue as strength is changed. This includes cyan and magenta printing inks.

However, it can be seen that the orientation of the DE₀₀ ovoid in C*L* does not orient with the data in orientation. This is soooo typical of C*L* ovoids!

So, DE₀₀  from the average is a much better metric than DE₀₀ from target color. If you have nothing else to use, this is preferred. If you are reading this shortly after this blog was posted, and you aren't using my computer, then you don't have nothing else to use, since these wonderful algorithms have not migrated beyond my computer as I write this. I hope to change that soon.

Conclusion

For the purpose of conformance testing, there is no question that DE is the choice. DE₀₀ is preferred to ΔE_ab(or even DE_CMC  or DE₉₄  or DIN 99).

For the purpose of SPC -- characterizing your color process to outliers -- the DE from target metric is lousy. The use of DE from average is preferable, but the best metric is Zc, which is based on Color SPC and fitting ellipses to your data.

Munsell - the Father of Color Science? (part 3)

February 19, 2018, 6:00 am

≫ Next: Is my color process all wonky?

≪ Previous: Is my color process going awry?

$

0

0

This series of blogs was foretold in a prophecy of April of 2013:

Someday I will write a blog post about how this guy Munsell laid the foundation for the ever-popular color space CIELAB, and came to be known as the Father of Color Science. He was also the father of A. E. O. Munsell, who carried on his work. I don't intend to write a blog post about how Albert became the father of A. E. O.

What I did not foretell in that blog post is that ISCC will be sponsoring the Munsell Centennial Symposium,  June 10 - 15, 2018 in Boston. Or that I would be keynoting this event.

After two previous attempts (Munsell as an educator, Munsell and 3D color space), I am finally have made my way to looking at the most significant work of Munsell. The Munsell Color Space was a model for CIELAB.

First cursory pass

Exhibit A. Richard Hunter's book The Measurement of Appearance, on page 136.

Photo taken at the Color Difference family picnic

This is a family tree of proposed models for determining color difference. Note that the Munsell Color system is in the upper right hand corner, and all arrows come from that box. The only little boxes that are still active today are the two boxes labeled CIE 1976. A similar diagram is on page 107 of that same book, which shows a family tree of color scales. (I have an image of that in a previous blog posts about color difference.) Again, this shows a straight lineage from Munsell Color Scpce to CIELAB.

Is this reliable testimony? Richard Hunter was a fairly knowledgeable guy when it comes to color. I mean, he has his own entry in Wikipedia for goodness sake. CIELAB is (perhaps) the most widely used tool in the color industry. Since Hunter traces the lineage of CIELAB back to Munsell, then I feel pretty confident about putting Munsell on the shortlist of highly influential figures in the history of color science, at the very least.

But, that hides a lot of the fun stuff that happened between the creation of the Munsell color space and the ratification of CIELAB as a standard for color measurement.

What Munsell did

    Munsell Color Space

Munsell's color space is based on some simple principles.

1. Hue, Value, and Chroma

There are three attributes to color in Munsell's color system. While these are implicit in many of the previous color systems (enumerated in a previous blog post), Munsell was intent on tying these to our intuitive understanding of color. (After studying on this for 25 years, I have come to realize that they are indeed intuitive). 

2. A physical standard produced with simple tools, simple math, and a defined procedure

Munsell described the procedure by which his color system could be developed from any reasonable set of pigments. The procedure included a way to assign unique identifiers to each color.  As a result, all colors within the gamut of the chosen pigments could be unambiguously named.    

3. Perceptual linearity

One of Munsell's secondary aims was to create a color space where the steps in hue, value, and chroma were all perceptually linear. Did he meet his goal? Stay tuned.


This color system was used to create the Atlas of the Munsell Color System, which was a book containing painted samples with their corresponding designations of hue, value, and chroma. This book was to be used as an unambiguous way to identify colors, and thus, to provide a standrd way to communicate color.

     Munsell photometer and the gray scale

Munsell invented and patented a photometer which was capable of measuring the reflectance of a flat surface. Well, provided it was a neutral gray. The user would look into a box and see two things: the sample to be measured, and a standard white patch. The sample was illuminated with a constant illumination, and the white standard was illuminated with light through an adjustable aperture. To make a measurements, the size of the aperture was adjusted so as to match the intensity of the dimmed white standard and that of the sample. The width of the aperture, scaled from 1 to 10, was the Munsell Value for the gray sample.

A shoebox with some holes and stuff

Munsell used his photometer to mix black and white paints in steps from V = 1 to V = 10.

     Maxwell disks and the rest of the colors

James Clerk Maxwell invented a creature called the Maxwell disk around 1855. I spent the better part of a day building my own set of Maxwell disks from colored construction paper as shown below. The cool part is the slit. You can slide two or more disks together, and rotate them so as to get any proportion of the colors to show. In the inset, I show the device that I adapted to rotate the disks. Again, the better part of a day was spent assembling a bolt, a couple of washers, and a nut. I first tried a cordless drill, and found it didn't spin fast enough to merge the colors. I had to use my old drill that plugs into the wall.

The Maxwell disks were the inspiration for PacMan

The picture below shows the results of day 3 of my dramatic reenactment of Munsell's landmark experiment. I selected red, green, and blue construction paper, and adjusted the size of the segments in order to get a facsimile of gray. When I saw that gray, I realized that this was four days well spent.

Me, geeking out on the creation of gray from Red, green, and blue

If I were to be doing this on a government grant, I would have spent another day or two actually measuring the sizes of the red, green, and blue areas. For the purposes of this blog, I will be content with just saying that red and blue are each one-quarter, and green is one-half. In other words, this green is half as strong as the others. Thus, Munsell would conclude that the chromas of this red and blue were twice that of this green. Munsell would also have measured this gray with his photometer. Another opportunity for me to get a little more grant money.

In this way, Munsell was able to assign values to the colors.

     Perceptual linearity?

Linear in Value?

Since Munsell's original Value was measured as the width of an aperture, the amount of light let through is proportional to the square of the Value. Conversely, Value is proportional to the square root of the light intensity. The plot below compares this scale against today's best guess at perceptual linearity, CIEDE2000.

Munsell's original Value was kinda sorta close to perceptually linear

Note: The DE2000 scale in the plot above is based on Seymour's formula (L00 = 24.7 Log e (20 Y +1), where 0< Y < 1), which was first presented at TAGA 2015, Working Toward A Color Space Built On CIEDE2000. The height of the curve at the end shows that there are 76 shades of gray, based on DE2000. The Munsell Value has been scaled to that.

Is this perceptually linear? That depends on how gracious you want to be. On the one hand, the linearity is not lousy. Given the tools at hand, Munsell did a fairly decent job of making kinda linear.

On the ungracious side, Munsell merely took what he had handy (the size of the opening of his aperture) and used that. Lazy bum! Surely he would have known about the work of Ernst Weber (1834) and Gustav Fechner (1860) which postulated that all our perception is logartihmically based! 

Really pedantic note: There is some confusion about how the gray scale was set up. My description is based on Munsell's description [1905], as well as comments by Tyler and Hardy [1940], Bond and Nickerson [1940], and Gibson and Nickerson [1940], all of which were based on Munsell's words and measured samples. But in a paper from 2012, Munsell described his assignment of Value as being logarithmic, following the Weber-Fechner law.

Linear in hue?

Munsell started this exercise by selecting five paints with vibrant colors: red (Venetian red), Yellow (raw sienna), green (emerald green), blue (cobalt), and purple (madder and cobalt). He then created paints that were opposite hues for each of these. The opposite hues were adjusted so that the balanced out to gray on the Maxwell disks. Thus, he had a set of ten colors with Value of 5 and Chroma of 5.

What's to say that these paints are equally spaced in hue? I am sure that Munsell selected them with that in the back of his mind, but four of the five are just commonly available, single pigment paints.
From the literature that I reviewed in the bibliography below, I could find no evidence that he put much time into psychophysical testing.

I'm gonna say that the hue spacing in the original Munsell color system is only somewhat perceptually linear.

Linear in Chroma?

Munsell's assignment of Chroma values is all based on simple ratios of areas on the Maxwell disks. Thus, in his original system, chroma is linear with reflectance. I did a bit of testing, comparing Munsell's proposition against DE2000. I will smugly state that our perception is not linear with reflectance.

But Munsell begs to differ with me. He performed some tests of this, and summarized his results in 1909:

These experiments show clearly that chroma sensation and chroma intensity (physical saturation) vary not according to the law of Weber and Fechner, but nearly or quite proportionately, and in accordance with the system employed in my color notation.

This paper seems to have been largely ignored by other color researchers. Deane Judd looked at the question of equal steps in chroma in 1932. His bibliography included Munsell's 1909 paper, but he made no mention of it in the text. The same with several of the papers from 1940 listed below.

My brief test suggests this is not true, and the people who were genuinely interested in the question who were aware of Munsell's suggestion ignored it. The graphs from the 1943 paper (Newhall, et al.) are decidedly non-linear in steps of chroma. Barring further evidence, I would say that the original Munsell Color System was not perceptually linear in chroma.

All in all, I'm gonna rate the claim that the original Munsell system was perceptually linear as "Mostly False".

What happened after Albert Munsell

Albert Munsell passed on in 1918, but a lot of work was done on the Munsell Color System by others after his death.

In 1919 and again in 1926, Munsell's son, A. E. O. Munsell submitted samples to the National Bureau of Standards. These were measured spectrophotometrically. The 1919 data was analyzed by Priest et al., and came along with some suggestions for improvement. They suggested that the Value scale be changed. 

This challenge was taken up by Albert's his own son. In 1933, A. E. O. published a paper describing a modification of the function from which Value was computed. This brought value much closer into line with the predictions of CIEDE2000.

The Munsell Color System was largely ignored in the literature until 1940. At that time, seemingly everyone jumped on the bandwagon. A subcommittee of the Optical Society of America was formed, and the December 1940 issue of the Journal of the Optical Society in America published five papers on the Munsell Color System.

Why the sudden effort? Spectrophotometers were expensive and cumbersome, but were becoming available. The 1931 tristimulus curves were available to turn spectral data into human units. Several of the papers noted a desire to create a system which translated physical measurements into something that made intuitive sense.

The Munsell Color System seemed to be best template to shoot for, since it was "[l]ong recognized as the outstanding practical device for color specification by pigmented surface standards."  (Newhall, 1940)

The efforts of the OSA subcommittee culminated in what has become known as the Munsell Renotation Data, introduced in the 1943 paper by Newhall et al. Inconsistencies of the original data were smoothed out, a new Value scaled was introduced, and a huge experiment (3 million observations) was done to nudge the colors into a system that looked perceptually linear.  The final result is a color system that can indeed be said to be perceptually linear.

Oh what a tangled web we weave, from Newhall (1943)

I'm not gonna take up the rest of the story, from the Renotation Data to CIELAB. That's another long and interesting story, I'm sure. But I am running out of gas!

Conclusion

Here is the firmest entirely factual statement that I can make about this paternity suit involving Albert Munsell and the child named Color Science.

Munsell had a passion for teaching color, especially to children. He sought to bring order and remove ambiguity from communication of color. This passion brought him to create the Munsell Color System. This was not the first three-dimensional arrangement of color, nor was it all that close to being perceptually linear. But it had two great features going for it: It was built on the intuitive concepts of hue, chroma, and lightness, and it came with a recipe for building a physical rendition of the color space. As a result, the Munsell Color Space is both a concept for understanding color, and a physical standard to be used in practical communication of color.

The Munsell Color System saw a number of improvements after his death, resulting in the Munsell Renotation Data. This later became the framework for future development of a magic formula to go from measured specrta to three numbers that define a color. The CIELAB formula is the one that stuck.

I realize that my work over the past 25 years has given me a bias toward the importance of measurement of surface colors, and hence a bias toward thinking that CIELAB is important. The next statement is subjective, and based on my admitted biases.

I think that Albert Munsell deserves to be called The Father of Color Science.

Albert Munsell proudly showing off his very attractive John the Math Guy Award

On the other hand...

I would be remiss if I failed to mention a few other individuals, who might reasonably be on the podium with Munsell.

Isaac Newton - He invented the rainbow, right? Well, actually, he did some experiments with light and came up with the theory of the spectrum. Spectrophotmeters are designed to measure this.

Thomas Young - He first proposed the theory that the eye has three different sensors (red, green, and blue) in 1802. Hermann von Helmholtz built on this in 1894.

Ewald Hering - He proposed the color opponent theory in 1878. Light cannot be both red and green; nor can it be both blue and yellow. His three photoreceptors were white versus black, red vs green, and yellow vs blue. This is explicitly built into CIELAB.

It turns out that all of these are correct, but they are looking at different stages in our perception. Newton's spectrum is a real physical thing. The retina does have three Young-Helmholtz sensors. The cones are not exactly RGB, but kinda. And the neural stuff after the cones in the retina creates signals that follow Hering's theory.

So, maybe one of these gents should get the crown? I dunno... maybe I'll make a few more John the Math Guy awards?

Bibliography

Munsell's papers

Munsell, Albert H., A Color Notation, Munsell Color Company, 5th Edition, 1905, Chap V

Munsell, Albert H., On the Relation of the Intensity of Chromatic Stimulus (Physical Saturation) to Chromatic Sensation, Psychological Bulletin, 6(7), 238-239 (1909)

Munsell, Albert H, A Pigment Color System and Notation, Amer. Journal of Psych, Vol 23, no. 2, (April 1912)

Post Munsell, pre-1940

Priest, Irwin, K. S. Gibson, and H. J. McNicholas, An examination of the Munsell color system. I. Spectral and total reflection and the Munsell scale of value, Tech. Papers of the Bureau of Standards, No. 167 (September 1920)

Judd, Deane, Chromatic Sensibility to Stimulus Differences, JOSA 22 (February 1932)

Glenn, J. J. and J. T. Killian, Trichromatic analysis of the Munsell Book of Color, MIT Thesis (1935), also in JOSA December 1940

The 1940's flurry

Gibson, Kasson S and Dorothy Nickerson, An Analysis of the Munsell Color System Based on Measurements Made in 1919 and 1926, JOSA, December 1940

Newhall, Sidney, Preliminary Report on the O.S.A. Subcommittee on the Spacing of the Munsell Colors, JOSA, December 1940

Tyler, John E. and Arthur C. Hardy, An Analysis of the Original Munsell Color System, JOSA December 1940

Nickerson, Dorothy, History of the Munsell Color System, Company, and Foundation. II. Its Scientific Application, JOSA, December 1940

Bond, Milton E., and Nickerson, Dorothy, Color-Order Systems, Munsell and Ostwald, JOSA, 1942

Newhall, Sidney M., Dorothy Nickerson, and Deane B. Judd, Final Report of the O.S.A. Subcommittee on the Spacing of the Munsell Colors, JOSA July 1943

More recent

Hunter, Richard S., The Measurement of Appearance, John Wiley, 1975, pps. 106 - 119

Is my color process all wonky?

February 28, 2018, 6:00 am

≫ Next: Scientists discover new astrological sign

≪ Previous: Munsell - the Father of Color Science? (part 3)

$

0

0

In a previous post, I looked at how the Zc statistic can be used to isolate individual color measurements that are icky-poopy. Today I look at a slightly broader question: How can I tell if the whole production run is wonky?

I think something went wrong in this production run

I deliberately use the word "wonky" because it has a squishy meaning, which is helpful, since I'm not sure what I want it to mean just yet! So, bear with me while I fumble around in the darkness, not knowing quite what I am doing just yet. 

Here is the premise: Color, in a well-controlled production run, should vary according to some specific type of statistical distribution. (Math mumbo-jumbo alert) I will take a guess that the cloud of points in that ellipsoid of L*a*b* values is a three-dimensional Gaussian distribution, with the axes appropriately tilted and elongated. If this is the case, then the distribution of Zc will be chi with three degrees of freedom. (End math mumbo-jumbo alert.)

If you are subscribed to the blog post reader that automatically removes sections of math mumbo-jumbo, then I will recap the last paragraph in a non-mumbo-jumbo way. In stats, we make the cautious assumption of the normal distribution. Since I am inventing this three-dimensional stats thing, I will cautiously assume the three-dimensional equivalent. But, since this is virgin territory, I will start by testing this assumption.

A quick note about CIELAB target values and DE

This blog post is not about CIELAB target values and DE. Today, I'm not talking about assessing conformance, so DE is not part of the discussion. I am talking about whether the process is stable, not whether it's correct.

A look at some real good data

Kodak produced a photographic test target, known as the Q60 target, which was used to calibrate scanners. The test target would be read by a scanner, and the RGB values which were read were compared against L*a*b* values for that batch of targets in order to calibrate the scanners. When the scanner encountered that same type of film, this calibration would be used to convert from RGB values to moderately accurate L*a*b* values. Hundreds of thousands of these test targets were manufactured between 1993 and 2008.

I think the lady peeking out on the right is sweet on me

We know that these test targets were produced under stringent process control. They were expensive, and expensive always means better. More importantly, they were produced under the direction of Dave McDowell. I have worked with him for many years in standards groups, and I know they don't come more persnickety about getting the details right than him!

Dave provided me with data on 76 production runs of Ektachrome, which was averages of  the L*a*b* values from 264 patches, for a total of about 20K data points. So, I had a big pile of data, collected of production runs that were about as well regulated as you can get.

I applied my magic slide rule to each set of the 264 sets of 76 color values. Note that I pooled at the data for individual colors of patches. General rule in stats: You don't wanna be putting stuff in the same bucket that belongs in different buckets. They will have different distributions.

Within each of the 264 buckets, I computed Zc values. Twenty thousand of them. I hope you're appreciative of all the work that I did for this blog post. Well... all the work that Mathematica did.

Now, I could have looked at them all individually, but the goal here is to test my 3D normal assumption. I'm gonna use a trick that I learned from Dave McDowell, which is called the CPDF.

Note on the terminology: CPDF stands for cumulative probability density function). At least that's the name that it was given in the stats class that I flunked out of in college. It is also called CPD (cumulative probability distribution), CDF (cumulative distribution function), and in some circles it's affectionately known as Clyde. In the graphic arts standards committee clique, it has gone by the name of CRF (cumulative relative frequency).

Here is the CPDF of the Ektrachrome data set. I through all the Zc values into one bucket. In this case I can do this. They belong in the same bucket, since they are all dimensionless numbers... normalized to the same criteria. The solid blue line is the actual data. If you look real close, you can see a dotted line. That dotted line is the theoretical distribution for Zc that 3D normal would imply. Not just one particular distribution -- the only one.

20,000 color measurements gave their lives for this plot

Rarely do I see real world data that comes this close to fitting a theoretical model. It is clear that L*a*b* data can be 3D normal.

More real world data

I have been collecting data. Lots of it. I currently have large color data sets form seven sources, encompassing 1,245 same-color data sets, and totalling 325K data points. When I can't sleep at night, I get up and play with my data. 

[Contact me if you have some data that you would like to share. I promise to keep it anonymous. If you have a serious question that you want to interrogate your data with, all the better, Contact me. We can work something out.] 

I now present some data from Company B, which is one of my anonymous sources. I know you're thinking this, but no. This is not where the boogie-woogie bugle boy came from. This complete data set includes 14 different printed patches, sampled from production runs over a full year. Each set has about 3,700 data points. 

I first look at the data from the 50% magenta patch, since it is the most well-behaved. The images below are scatterplots of the L*a*b* data projected onto the a*b* plane, the a*L* plane, and the b*L* plane. The dashed ellipses are the 3.75 Zc ellipses. One might expect one out of 354 data points to be outside of those ellipses.

Three views of the M 50 data from Company B

Just in case you wanted to see a runtime chart, I provide one below. The red line is the 3.75 Zc cutoff. There were 24 data points where Zc > 3.75. This compares to the expectation of 10.5. This is the expectation under the assumption that the distribution is perfectly 3D normal. I am not concerned about this difference; it is my expectation that real life data will normally exceed the normal expectations by a little bit.

Another view of the M 50 data - Zc runtime plot

So far, everything looks decent. No big warning flags. Let's have a look at the CPDF. PArdon my French, but this looks pretty gosh-darn spiffy. The match to the theoretical curve (the dotted line) is not quite as good as the Ektachrome data, but it's still a real good approximation. 

Another great match 

Conclusion so far, the variation in color data really can be 3D normal!

Still more real world data

I show below the CPDF of Zc for another data set from that same source, Company B. This particular data set is a solid cyan patch. The difference between the real data and the theoretical distribution is kinda bad.

A poor showing for the solid cyan patch

So, either there is something funky about this data set, or my assumption is wrong. Maybe 3D normal isn't necessarily normal? Let's zoom in a bit on this data set. First, we look at the runtime chart. (Note that this chart is scaled a bit different than the previous. This one tops out at Zc = 8, whereas the other goes up to 5.5.) 

A runtime chart with some aberrant behavior
that will not go unpunished!

There are clearly some problems with this data. I have highlighted (red ellipse) two short periods where the color was just plain wonky. Some of the other outliers are a bit clustered as well. Below I have an a*b* scatter plot of that data (on the left), and a zoomed-in portion of that plot which shows some undeniable wonk. 

Look at all the pretty dots that aren't in the corral where they belong

  

I'm gonna say that the reason that the variation in this data set does not fit the 3D normal model is because this particular process is not in control. The case gets stronger that color variation is 3D normal when the process is under control.

Are you tired of looking at data yet?

We have looked at data from Company K and Company B. How about two data sets from Company R? These two data sets are also printed colors, but they are not the standard process colors. There are  about 1,000 measurements of a pink spot color, and 600 measurements of a brown spot color. One new thing in this set... these are measurements from an inline system, so they are all from the same print run.

First we look at the CPDF for the pink data. Yes! I won't show the scatterplots in L*a*b*, but trust me. They look good. Another case of "3D normal" and "color process in good control" going hand-in-hand.

Yet another boring plot that corroborates my assumptions

Next we see the CPDF of Zc for the brown data. It's not as good as the pink data, or the Kodak, or the M 50 CPDF plots, but not quite as bad as the C 100. So, we might think that the process for brown is in moderate control?

Brown might not be so much in control?

The runtime chart of Zc looks pretty much like all the others (I could plop the image in here, but it wouldn't tell us much). The scatter plots of L*a*b* values also look reasonable... well, kinda. Let's have a look.

Halley's comet? Or a scatterplot of variation in brown?

.This data doesn't look fully symmetric. It looks like it's a little skewed toward the lower left. And that is why the CPDF plot of brown looks a bit funky. Once again, we see that the CPDF of Zc values for a set of color variation is a decent way to quickly assess whether there is something wrong with the process.

But why is the brown plot skewed? I know the answer, but we're gonna have to wait for the full exposition in another blog post.

For the tine being, let me state the thrilling conclusion of this blog post.

The thrilling conclusion of this blog post

When a color producing process is "in control" (whatever that means), the variation in L*a*b* will be 3D normal. This means that we can look at the CPDF of Zc as a quick way to tell if we have exited the ramp to Wonkyville.

Scientists discover new astrological sign

April 1, 2018, 12:27 pm

≫ Next: A Tale of capitalism with a little twist

≪ Previous: Is my color process all wonky?

$

0

0

NASA scientist Ben Capricorn announced today the discovery of a thirteenth sign of the zodiac, which has been tentatively named Naivius the Confounder. The constellation for this sign differs from all others in that it spans the entire 360 degree sky.

Naivius the Confounder is obvious, once you see it

Dr. Capricorn explained that he was studying anomalies in horoscopes, people who did not match their signs. "I had been pondering a blog post by John the Math Guy which showed that the signs of the zodiac are useless in predicting mathematical genius. It suddenly occurred to me that there must be another celestial influence which has some effects that were not seen by Ptolemy, who codified these laws a millennium or so ago." Dr. Capricorn theorized that the influence must have been from heavenly bodies that were not known at the time of the discovery of astrology.

Dr. Ben Capricorn of NASA's Office of Space

So, Capricorn petitioned NASA for time on the Hubble telescope to peer deeply into the twelve constellations to find occult stars that might explain the anomalies. Simon Rasputin, director of the Government Office of Pseudoscience applauds this effort. "Capricorn's work is far beyond anything that cosmologists have been able to piece together with all their silly-talk about black holes and the red shift and the big bang and all that stuff. It make so much more sense to group stars that are billions of light years apart into constellations [rather than group them according to the groups of stars that are near together and which rotate about a common center of mass.]  Mystical forces are way more better than dumb equations."

Capricorn's theory was proven true. He was able to find minor stars which correspond to each of the anomalous mathematicians who were not Virgos, as all real math guys should be. These minor stars together form the constellation Naivus the Confounder.

This confirmation of Dr. Capricorn's theory is just the start of this momentous task. "I have already applied for a grant to continue this work. For the project, I will investigate the horoscopes of each and every one of the 7.4 billion people on the Earth, and find an occult star among the hundreds of billions of stars to explain why 91.7% of all people don't fit their horoscope. The project is staggering in it's proportions, but the ultimate benefit to humankind is immeasurable."

A Tale of capitalism with a little twist

April 3, 2018, 6:00 am

≫ Next: Expanded gamut - when an idea's time has come

≪ Previous: Scientists discover new astrological sign

$

0

0

Spoiler alert - this story has a surprise ending. That's all I'll say for now.

My family had a Monopoly board when I was growing up. I was the youngest of three, so I got used to ending the game in tears of frustration and humiliation when I landed on Pacific Avenue and had to surrender all my cash and properties over to my greedy sister.

I always lost. It seems I had some very bad character traits. When I landed on Boardwalk, and saw that it would cost $350 of the $600 that I had in front of me, I would opt for frugality. And when my sister pointed out that Illinois Avenue only cost me $240, and she was willing to pay me $300, I opted to be trusting and sell it to her. In the very unlikely event that I had developed a piece of property that my sister landed on, I was often lenient, letting her off with the rent for the undeveloped property. Frugality, trust, and compassion. Poor life choices on my part, indeed.

It looks like I may not be collecting that $200

(I have a sneaking suspicion that my sister may have a somewhat different memory of our board games. Whatever she tells you, just remember that I was the injured party.)

As I grew up, I eventually learned that the best Monopoly strategy was to buy every property that I possibly could, mortgage myself to the hilt to buy properties, and to be ruthless when it came to bargaining with other players. And here's an advanced tip: When all the properties have been purchased, it is better to sit in jail and collect rent rather than use the Get out of Jail Free card. All very important life skills that I learned from Monopoly. I had been indoctrinated into capitalist society.

The History of Monopoly

Everything I know about capitalism I learned from Monopoly

Monopoly was and is an immensely popular game. According to Time Magazine, "Monopoly is the most popular board game in history, with more than 250 million copies sold." If you don't believe Time, the Parker Brothers' website says that Monopoly is "the world's favorite family game brand!" I mean, Parker Brother's should know about their own game!

(Warning: the following few paragraphs contain some information which, for want of a better phrase, are outright lies. I promise to clear it up.)

The story of how this game came to be is a real-life rags-to-riches kinda story. Monopoly was invented by Charles Darrow during the Great Depression. You know... the one caused by ruthless investors who realized that their best strategy was to buy every stock they possibly could? Darrow had lost his job in sales, and pitched the game to both Milton Bradley and Parker Brothers. Initially, they both turned it down.

Ever resourceful, Darrow manufactured the game on his own and sold a very respectable number of copies during the 1934 Christmas season. His sales were enough to bring Parker Brothers back to the bargaining table. Parker Brothers eventually purchased the game and helped him get a patent. Darrow became the first person to become a millionaire by designing a board game. 

Proof that Darrow invented Monopoly

There we have it. The game of monopoly working out in real life. A resourceful and tenacious inventor with a good idea gets rich.

Some backstory

Remember how I said there would be a surprise ending? Before you get all warm and fuzzy about the wonderful Charles Darrow, I need to fill in a bit more of the backstory. Then I will get to the surprise ending.

First bubble to burst. Charles Darrow did not invent the game. It seems that Darrow and his wife Esther were invited to the home of Charles and Olive Todd in 1933 to play a board game. Darrow later bugged Todd for details on the game, and for a copy of the board and game cards. The game that Darrow sold to Parker Brothers was a direct copy of the game that Darrow badgered out of Todd. Darrow copied the game right down to a misspelling of Marven Gardens, in the game from Todd. Marven Gardens is a subdivision in Margate City, NJ, which abuts Ventnor City, but you will note that in Monopoly, it is spelled Marvin Gardens.

Marvin versus Marven

I don't want to imply that there was anything wrong with Darrow manufacturing a game that was in the public domain, or selling that game to Parker Brothers for a big bunch of money. This is capitalism in action. The fact that the Todds never again invited the Darrows over for Saturday night games was just sour grapes.

But...

Darrow told Parker Brothers that he had invented the game. That was a lie, which is morally icky. It's also a bad business practice to lie to business partners (my opinion), and it opened Darrow up for the possibility of a civil lawsuit from Parker Brothers. I do not consider this to be an example of capitalism at its finest.

Here is Darrow's description of the invention of Monopoly:

"Friends visiting our house in the later part of 1931 mentioned a lecture course they had heard of in which the professor gave his class scrip to invest and rated them on the results of their imaginary investments. I think the college referred to was Princeton University.

Being unemployed at the time, and badly needing anything to occupy my time, I made by hand a very crude game for the sole purpose of amusing myself.

Later friends called and we played this game, unnamed at that time. One of them asked me to make a copy for him which I did charging him for my time four dollars. Friends of his wanted copies and so forth."

Parker Brothers got a bit concerned when one of their VPs recalled a patent from 1904 for a very similar board game. Rather than sue Darrow, they asked Darrow to sign an affidavit to the effect that he was the rightful inventor of Monopoly. He signed it, thereby covering Parker Brothers' butt. 

Note that Darrow also filed for a patent of the game of Monopoly, which is legally a statement of inventorship. With these two legal documents -- the affidavit and the patent application -- Darrow  crossed the line to what I think is criminal activity. Now, I'm not a lawyer, so I can't give legal advice. All I can do is give illegal advice, and write that advice on an illegal pad. But here is my illegal advice: please check with your lawyer before signing any legal documents that you know to be lies. You don't want to be sent to bed without your supper.

Real computer programmers don't like goto statements

Tracing Monopoly backward

The Todds didn't invent Monopoly either, and they never claimed to. In sworn testimony, Charles Todd said that he learned the game from a classmate, Eugene Raiford, who in turn, learned the game from his brother Jesse Raiford.

Jesse was a real estate agent in Atlantic City. Of course, you know that all the properties in Monopoly are named after places in Atlantic City? Jesse Raiford is the guy who was responsible for putting reasonable purchase prices and rents for all the properties, based on his knowledge of the actual streets in question. So, when you buy Indiana Avenue for $220, and have to pay out $1300 when you land on Park Place with four houses, you can thank (or blame) Jesse. 

But he did not select which Atlantic City streets (from Baltic to Boardwalk) were immortalized in Monopoly. A teacher at the Quaker School by the name of Ruth Hoskins had learned of a game on a trip to Indiana. She brought the game back to Atlantic City, where the Quakers adapted the game to the Atlantic City neighborhoods that they were familiar with.

A detailed description of the history of the game prior to Atlantic city can be found elsewhere. I will content myself to jump all the way back to the square labelled "Go".

The Landlord's Game

In March of 1903,  Lizzie Magie filed for patent for what what she called "the Landlord's Game". The drawing below, from the patent, is of the game board. It shows that many of the aspects of the game that we all know and love data back to 1903. The upper left hand square instructs one to "GO TO JAIL". Directly opposite of this, in the lower right corner, is the jail. The square that we sophisticated 21st century beings call "FREE PARKING" was called "PUBLIC PARKING" in the original.

The original Monopoly board

Between each of the corners, we see nine spaces (just like today's Monopoly board), most of which have a sale price and a rent price. The middle space on all four rows is a railroad - exactly the same as the modern board. Utilities (light and water) each have a space, and the original board has not one, not two, but three spaces where one had to pay for luxury. 

The square that we call "GO" bears an odd label in the original version: "Labor upon Mother Earth Produces Wages". This is the square where the player receives his wages for "perform[ing] so much labor upon mother earth". 

Much of the play proceeds quite similar to the modern version, with the winner being the one with the most money after a predetermined number of trips around the board.

Pausing for some perspective

It is patently obvious that the game that Charles Darrow sold to Parker Brothers was largely derived from the 1903 patent by Lizzie Magie. Clearly Darrow lied about how the game was invented. There can be no question about that. But there are two other pertinent questions to address before I continue.

Did Parker Brothers violate Magie's patent?

No. At that time, patents lasted for 17 years after they were granted. (Today, the term is 20 years after the patent is filed.) This patent was granted in 1904, so it expired in 1921. After that time, the invention in the claims entered into the public domain, so anyone was free to make or sell Magie's board game in 1934.

Does the Magie patent invalidate Darrow's patent? 

It would be natural to think that, once "Monopoly" has been patented, the game is over, and it can't be patented again. But one of the cool things about patents is that the patent office is cool with you filing for a patent on an improvement to an existing patent. They even encourage it. Your improvement just can't be obvious when compared with any prior art.

The Magie patent does not describe naming of the properties, grouping properties by color, or rents that depend on additional investments into a property, or the need to own all of the same-colored properties before developing the property. So, if the patent examiner found this to be non-trivial, then there was room for additional patents. I readily admit that I have not spent the week or so necessary to fully understand the claims in Darrow's patent.

Magie's second patent

Elizabeth Magie Phillips (AKA Lizzie Magie) filed for a second patent on her Landlord's Game in 1923. There are some changes to the game. While the new properties have been given fictitious street names, the layout of the board departs from the original patent. The current Monopoly board is clearly derived from Magie's first patent, not from her second.

There are some additions which have made this filing patentable over the previous patent. The new set of claims includes the notion of a grouping of properties together. I would presume that this is a novelty which distinguishes the two patents. But again, I fully admit to not investing a lot of time into interpreting one set of claims over the previous disclosure. 

Magie's second patent was granted in 1924, which means that it was in effect when Parker Brothers and Darrow were signing their deal. Uh-oh. But, Parker Brothers did their due diligence, and negotiated with Magie over the rights for her second patent. She received $500.

That number might just tick you off, but hear me out...

The Monopoly game we all know and love is based on Magie's original patent, so you may feel that she has a right to any and all profit from the Monopoly game. But, she had her chance, between 1905 and 1921 to have a monopoly on Monopoly.

Another consideration is that $500 is not all that much money today. Adjusted for inflation, that $500 would be worth $9,300 today. Not a bad chunk of money. Considering the cost and effort of developing a successful product from a good idea, maybe the size of the check was reasonable?

A final consideration is that there is a real question whether the existing Monopoly game would infringe on the 1924 patent. As with all patents, it comes down to how to interpret all the elements of a claim. For example, claim 1 of the 1924 patent includes "a series of cards of changeable value, two or more of which are alike and which relate to two or more certain spaces on the board". If Monopoly can be shown to not have such changeable cards, then it does not infringe claim 1. To be honest, I don't know what the phrase means. Given enough time, I'm sure I could work up an argument either for Parker Brothers or for Magie.

So, Magie may have been able to parlay this patent into a lot more money, but probably not without  fair amount of legal expense and a certain risk. Collecting $9,300 may well have been a reasonable choice.

And that's capitalism for you.

One more thing...

One of the topics of this blog post is capitalism. I would be remiss if I didn't share a bit more about Elizabeth Magie and some wording from her second patent.

Magie describes the purpose of the game as follows: "The object of the game is not only to afford amusement to the players, but to illustrate to them how under the present or prevailing system of land tenure, the landlord has an advantage over other enterprises and also how the single tax would discourage land speculation".

Here are some other political gems in the patent.

1. There is a space on the board named "Lord Blueblood's estate". "This space represents foreign ownership of American soil, and carries with it a jail penalty for trespassing."

2. Another space is call "La Swelle Hotel". "This space represents the distinction made between classes, only moneyed guests being accepted." 

3. In one particular misfortunate toss of the dice, the player will have been caught robbing the public. They will take $200 from the bank, and the other players will thereafter call the player "Senator". To the best of my knowledge, this is the harshest insult to be found in all the patent archives.

Clear political undertones. And overtones too, for that matter!

Magie's intent with the game was to illustrate the inequity inherent in the idea of people getting paid by rent, beyond pay for actual labor. In a sense, you could say that she was using the game to indoctrinate the players into this anti-capitalistic idea. You see, Elizabeth Magie was a Georgist, one who believes that people should earn money from hard work, but not from property or natrual resources.

The irony is that the game Monopoly has been used for several generations as a training mechanism for young capitalists. That's my surprise ending.

References

Cheating at Monopoly: Uncovering the secret history of the classic board game
http://www.nj.com/entertainment/index.ssf/2015/03/the_monopolists_mary_pilon_monopoly_atlantic_city.html

Stealing Monopoly
https://joemckinney.wordpress.com/2009/10/19/stealing-monopoly/

The Culture Complex: Monopoly Is Us
http://content.time.com/time/subscriber/article/0,33009,1535818-1,00.html

MONOPOLY: From Berks to Boardwalk
https://landlordsgame.info/articles/berks2boardwalk.html

The fake history — and the real one — behind the inventing of ‘Monopoly’
https://www.washingtonpost.com/news/book-party/wp/2015/02/18/the-fake-history-and-the-real-one-behind-the-inventing-of-monopoly/?utm_term=.a7bdcbc1da3c