Difference between revisions of "stat340s13"

From statwiki
Jump to: navigation, search
Line 174: Line 174:
#let λ=2 in this example; however, you can make another value for λ
>>size(x)      #1000 in size  
>>size(x)      #1000 in size  
Line 184: Line 185:
=P(F^(-1)(U)<=x)<br />
=P(F^(-1)(U)<=x)<br />
=P(F(F^(-1)(U))<=F(x))<br />
=P(F(F^(-1)(U))<=F(x))<br />
=P(U<=F(x))<br />
=P(U<=F(x)), where U~Unif[0,1]<br />
<!-- Did class end before this was finished? -->
<!-- Did class end before this was finished? -->

Revision as of 22:22, 9 May 2013

Computer Simulation of Complex Systems (Stat 340 - Spring 2013)

Introduction, Class 1 - Tuesday, May 7

Four Fundamental Problems

1. Classification: Given an input object X, we have a function which will take in this input X and identity which 'class (Y)' it belongs to (Discrete Case)
2. Regression: Same as classification but in the continuous case
3. Clustering: Use common features of objects in same class or group to form clusters.(in this case, y is unknown)
4. Dimensionality Reduction


Most useful when structure of the task is not well understood but can be characterized by a dataset with strong statistical regularity

  • Computer Vision, Computer Graphics, Finance (fraud detection), Machine Learning
  • Search and recommendation (eg. Google)
  • Automatic speech recognition, speaker verification
  • Text parsing
  • Face identification
  • Tracking objects in video
  • Financial prediction, fraud detection

Course Information

General Information

  • No required textbook, recommended: "Simulation" by Sheldon M. Ross
  • Computing parts of the course will be done in Matlab, but prior knowledge of Matlab is not essential (will have a tutorial on it)
  • Learn for announcements, assignments, and emails.
  • Other course material on: http://wikicoursenote.com/wiki/
  • Log on to both Learn and wikicoursenote frequently.

Primary contributor: Put a summary of the lecture up within 48 hours.

General contributor: Elaborate on concepts, add example, add code, add pictures, reference, corrections etc… withing 2 weeks

Must do both All primary contributions are now considered general contributions you must contribute to 50% of lectures for full marks

  • A general contribution can be correctional (fixing mistakes) or technical (expanding content, adding examples, etc) but at least half of your contributions should be technical for full marks

Do not submit copyrighted work without permission, cite original sources. Each time you make a contribution, check mark the table. Marks are calculated on honour system, although there will be random verifications. If you are caught claiming to contribute but didn't, you will lose marks.

Tentative Marking Scheme

Item Value
Assignments (~6) 30%
WikiCourseNote 10%
Midterm 20%
Final 40%

Sampling (Generating random numbers), Class 2 - Thursday, May 9


Some people believe that activities such as rolling a dice and flipping a coin are not truly random but are deterministic – that is, the result can be reliably calculated using things such as physics and math.

A computer cannot generate truly random numbers since computers can only run algorithms, which are deterministic in nature. They can, however, generate Pseudo Random Numbers; numbers that seem random but are actually deterministic.

Multiplicative Congruential Algorithm

This is an algorithm used to generate uniform, pseudo random numbers. It is also referred to as the Linear or Mixed Congruential Methods. The Multiplicative Congruential Method may also refer to the special case where c=0.

Take a number x and divide it by m
The remainder is a number between 0 and m-1
We use the operator “mod”
X mod m
1 = 10 mod 3
x = x mod m – we take the remainder and go remainder mod m, but we will get x every time we do this
We can modify this:
Z = (ax+b) mod m

Example: a=2, b=1, m=3; if x=10
Step 1: 0 = (2(10)+1) mod 3
Step 2: 1 = (2(0)+1) mod 3
Step 3: 0 = (2(1)+1) mod 3
You will get a sequence of numbers

MatLab for Multiplicative Congruential Algorithm:
Before your start:

>>Clear all
>>Close all

(Note: Keep repeating this command over and over again and you will seem to get random numbers – this is how the command rand works in a computer.)

>>for ii=2:1000
ans=1    1000

MCA Example.jpg

(Note: The semicolon after the x(ii)=mod(a*x(ii-1)+b,m) ensures that Matlab will not show the entire vector that is x. It will instead calculate it internally and you will be able to work with it. Adding the semicolon to the end of this line reduces the run time significantly.)

This algorithm involves three integer parameters a, b, and m and an initial value, x_0 called seed. A sequence of numbers is defined by x(k+1) = a*x(k) + b mod m. Mod m means take the remainder after division by m.

Example: a=13, b=0, m=31
The first 30 numbers in the sequence are a permutation of integers for 1 to 30 and then the sequence repeats itself. Values are between 0 and m-1. If the values are normalized by dividing by m-1, then result is numbers uniformly distributed in the interval [0,1]. There are only a finite number of values (30 in this case).

Typically, it is good to choose m such that m is large, and m is prime. Careful selection of parameters helps generate relatively "random" output values, where it is harder to identify patterns.

For many years “rand” function in Matlab used this algorithm with these parameters A=7^5=16807, b=0, m=2^31 -1=2147483647 – recommended in a 1988 paper by Park and Miller (Important part is that m should be large)

Inverse Transform Method

This method is useful for generating other types of distributions other than uniform, such as exponential distribution and normal distribution. Exponential distribution have the property that numbers generated are more frequently close to 0. Normal distribution has the property that numbers generated are more frequently close to its mean.

Take U ~ U[0,1] and let x=F^(-1)(U)
Then x has distribution function F(.)
Where F(x) = P(X<=x) cdf; F^(-1)(U) denotes the function inverse of F(.) Or that F(x)=U -> x=F^(-1)(U)

Example: f(x) = λe^(-λx)
= ∫0tox〖λe^(-λx) dx〗
=λ/(-λ) e^(-λx) [0 to x]
=(-e^(-λx)+e^0 )

F^(-1) (x)=-ln(1-x)/λ;

Step 1: Draw U ~U[0,1];
Step 2: x=-ln(1-U)/ λ;

MatLab for Inverse Transform Method:

>>hist(u)       #will generate a fairly uniform diagram

ITM example hist(u).jpg

#let λ=2 in this example; however, you can make another value for λ
>>size(x)       #1000 in size 
>>hist(x)       #exponential 

ITM example hist(x).jpg

=P(U<=F(x)), where U~Unif[0,1]

This method is flawed since not all functions are invertible. The method is limited also because not all functions are monotonic. Also, it may be impractical since some CDFs and/or integrals are not easy to compute.

Probability Distribution Function Tool in MATLAB