RetroChallenge Day 15 – Post 7 – Anderson-Darling Test Part 2 – CDF Look Up

From the last post the formula for A-squared the test statistic for the Anderson-Darling normality test is :

cdfmtbl9

Where the part in the red box is the cumulative density function for the normal distribution.  This is the area under the normal curve from negative infinity to Yi.  This represents the probability that a variable takes on a values less than or equal Yi.  So this will be a value between 0 and 1.

cdfmtbl6

The values in the CDF table we loaded last time represent the area under the curve from 0 to Yi ( or x depending how you want to name), for a distribution with a mean of 0 and standard deviation of 1.  So to get the area from negative infinity an additional calculation is made.  If x>0 then 0.5 is added to the table value, if x is less than 0 then 1-table value is used.  These examples will clarify, :

cdfmtbl10

cdfmtbl8

So this is great if every distribution had a mean of 0 and standard deviation of 1.  Luckily there is an approach that allows us the use the table for test any sample of data, and it is described in the green box above.  Yi = (xi – xbar) / standard deviation

Using the table created in the last post here is the procedure (in the file STATAD.P) to lookup the table value of a data point X, from a sample with a mean of XBAR and standard deviation of STD, and then calculate the CDF value.   The value is returned in the variable LU, from a call LOOKUPCDF(x, xbar, std, lu);

lookup1

A quick code review

Green arrow : standardize value of X to use the table;
Red arrow :   multiple R by 100 to convert to the table index and store in J
(note: the table is indexed from 0, and is in singled dimensioned unlike the physical table, which has the two lookup values)
Blue Arrow: determine if J is greater or less than 0 and set the pointer to lookup in the table.
Yellow arrow : read the CDF.TBL file using the index K
Gray arrow : using the value looked up and if J is positive or negative make the appropriate adjustment to the table value (add 0.5 J > 0 or subtract from 1 if j<0)

lookup2

Then the calling procedure can use the value returned in the LU position in the call.

One note here is that this method of opening and reading the table does slow down the calculation, but on the other hand does not require the table to take up valuable memory.  Thus the challenges of our retro computing days, sometimes we had to choose between fast and memory…

The next post will show how this value is used to calculate A-Squared, A-Squared Prime and the P-value for the Anderson-Darling Test.

statad01

Advertisements

RetroChallenge 2017/04 Day 2 – Post 2 Dorsett Tape #2 – Graphs

Dorsett Educational Systems – Statistics Tape 2 covers graphs for data in frequency tables.  Frequency histograms, frequency polygons, cumulative frequency histograms and cumulative frequency polygons are reviewed.  In my modern use of statistics for process improvements we typically use histograms with a normal distribution superimposed on top.  This tape two run time was about 11 minutes a bit shorter than tape one.  Histograms will likely be coded into my version of statistics program.  Shown below are a set of screen shots from the lesson.  It is interesting to note that it is all done in text mode.

Next up :

Starting the statistics program in Kyan Pascal, and Tape 3 – Measures of central tendency.

Retrochallenge 04/2017 – Day 1 – Dorsett Tapes Post 1

Retro Challenge Day 1

A brief intro I guess would make sense, I’m Tom Raidna (@TRaidnaComputes on twitter)
and this is my fourth run in the RetroChallenge. For my first project I wrote a baseball game for Timex Sinclair 2068 / Spectrum, for the second run I wrote a Visual Basic IDE for the CC65 C compiler for Commodore 64, 128, Apple II, Atari 400/800 and Oric Atmos, the third run was to set up a Commodore128 and do some programming.  The first two projects were recognized by the judges, the third was a DNF, did not finish, sometimes life……

My background is in Math and Science education, and after teaching for 4 years I went back to school for Computer Science earned another BS degree, and then 20 years later, I went back to school again for a graduate certificate in Applied Statistics to support my work in Lean Six Sigma (process improvement methodology) , thus the combination of programming, Dorsett Stats learning for this RetroChallenge.
So onto the project:

To document this project I will post separately for the review of the Dorsett Tapes and for the STATS program that will be written in Kyan PASCAL for Atari 400/800.  This will be the first post for the Dorsett Tapes course in statistics.

In order to set up to use the Atari, Altirra Emulator, Dorsett Educational System Tapes for Statistics, I downloaded the wav files in a zip file from archive.org

https://archive.org/details/DorsettAtariStatistics

And one also need to have the Education System Master Cartridge (Thanks Keven Savetz for the guidance here or I’d still be trying to figure out to use the wave fiels)

The cartridge can be found here :

http://www.atarimania.com/game-atari-400-800-xl-xe-educational-system-master-cartridge_1797.html

Once downloaded to both the master cartridge and individual cassette wave files need to be loaded in the emulator (just like putting the cartridge in the Atari 800 and then cassette in the tape drive).

Starting with File / Attach Cartridge…

and then select the Education Master Cartridge

tape0 attach cart

tape0 educational system master cartridge

 

 

 

 

 

Next the cassette (.wav file) can be loaded by selected File / Cassette / Load from the Altirra menus and selecting the cassette .wave file, st1.wav in the picture for the first tape in the course.

tape0 load cassette

tape0 load cassette 2

Now by resetting the emulated computer with Shift / F5 the system will boot to the Education Master startup and the screen below will be seen:

tap1 start screen

To access the Tape control to press plan use File  / Cassette / Tape Control and then press the > play button.  Then use the F2 Key (Start) to start the lesson.

While this seems like it might be a lot to just get started, written out, it’s really quite simple.  Now one can navigate the lessons.  In general the lesson flow is there is a voice over on screens of material.  To advance to the next screen a key press is needed, or there may be a knowledge check and correct answer to a multiple choice.

Tape 1 – Frequency Distributions

For each tape I’ll post a brief outline of the topics covered and a couple of screen shots, and I will make a YouTube video of part of one lesson as an example.

Tape 1 Topics

tape1 title

  1. Observations
  2. Population vs Sample
  3. Finite vs Infinite populations
  4. Random vs Biased Samples
  5. Continuous and Discrete data types
  6. Frequency distributions (Tables)
  7. Class frequency
  8. Class boundaries, lower and upper, class intervals, Interval width
  9. Frequency and cumulative frequency

Sample Screens:

This lesson gave an good introduction to stats and the concepts listed in the outline.  The one thing that caught my eye, is the multiple colors on the text screen, I’ll be interested into looking into how this was done (time permitting), as I thought only one color, hue combination was allowed in text mode.  (There may be some text windowing being used).  Frequency Histogram are planned for the STATS program.

Next posts will cover the introduction to the STATS program and Tape 2.

 

RetroChallenge 04/2017 – Entry

I’ve just submitted my entry for the April 2017 Retrochallenge and my entry is outlined below:

Using Altirra the Atari 400/800 emulator complete the following :

position_01

a. complete the Dorsett Education program (cassette s/w with voice) on Statistics – 16 cassette files downloaded from archive.org and document at traidna@wordpress.com
The files are wav format created from the cassettes as pictured below from the archive.org download.


b. Write a statistics program in Kyan Pascal on Atari 800 (emulator) which will compute
descriptive stats, Anderson-darling test for normality, one and two sample T-Tests, one and two proportion tests etc.

uk02
c. create YouTube.com videos on creating the statistics program, as there does not appear to be any for Kyan Pascal on Atari

d. blog on the process at traidna.wordpress.com, update the kyanpascal.wordpress.com blog on creating programs

Looking forward to my fourth Retrochallenge Campaign!!!