Friday, August 18, 2017

NodeMCU wireless IoTGarage 2.0 : Updated UI with AngularJS!

Hey everyone,
          I hope you had a great summer. I know I did. I had been very blessed to have been awarded with a scholarship from Google which also presented me with a flight and a 4 day retreat at the Google campus in Palo Alto, CA. It was an incredible experience in which i met many other amazing scholarship recipients as well as some very smart Google engineers. I also finished up a very rewarding 10 week internship at JPMorgan & Chase Technology. I learned so much while working there and was presented with all the help and encouragement I needed to succeed. They are definitely on the right track. With all that being said, I haven't forgot about my fellow Computer Science family out there and all the of you coders. I still maintain my will to grow my knowledge and that is what has led me this post. I decided to continue my learning on the previous garage server I have built and now I am updating it to IoTGarage 2.0. I decided to try my luck building the webpage as a stand alone AngularJS application to improve functionality and response time as well as user experience.  This all sounds nice, but it was mostly to learn and share my experience with all of you. Hopefully it helps to propel you forward in your learning.

The latest commit was pushed up to GitHub and can be found here: IoTGarage
I rewrote the server requests (other than the initial)  to respond to the Angular Http requests in text/json rather than loading all new html on each request. This is definitely a much better approach and gives the web page a much more native feel, like that of an application. It also leaves much room for growth on what information can be provided from the NodeMCU ESP8266 in the future. Information such as time between garage open/closings, past data that you chose to store about these actions, and much more. The webpage also now resizes based on which device you are accessing the server from. Therefore it is perfect for the mobile users. This makes sense due to the simple fact that most people wont be opening and closing their garages from their desktop..ha. Anyway, here are two looks at the new page on Desktop and then from a phone.  I hope this code helps everyone and you should feel free to message me about an problems or questions. Remember to always, SudoAptUpgradeYourBrain!

Here are a few pictures of the new UI.

                        When visited by desktop:




                           When visited by phone :



when open is click : 


When check is clicked : 


when closed is click : 




Wednesday, May 24, 2017

Create Your Own WiFi Garage Server Using NodeMCU esp8266

Hello everyone, I hope your summers are getting off to a great start. I recently kicked mine off with an IoT project. I'm here today because it is finally finished and I can put it out there for anyone else who needs help. The project I have for everyone today is a homemade Wifi connected garage server. It is a fairly inexpensive very intuitive project. The parts needed are as follows:



Breadboard 
Blue LED (or any color) 
White LED (or any color) 
1 - 330 ohm resistor
1 - 1k resistor

The Code, wiring diagrams (fritzing.com) can be found here: 

Node Schematic:

Magnetic Switch hooked up: 
the magnetic switch has a continuous  5 volts running through so that when the door is open, it cuts off the voltage. Then, gpio10 ( sdd3) on the board reads the low voltage and knows that the garage is open. This was a little tricky because even when the circuit was open, the board would read static voltage and would often not be able to tell the circuit was open. I solved this by adding a resistor, which says 1k but 330 ohms was fine, that would pull the static voltage to the ground when the circuit was open.


Relay switch to my garage door button: 
The original garage button runs a continuous voltage of 17volts, so I hooked up the 5v relay so that when the relay received a 5v high signal from the microcontroller, it closes off the voltage for 1 sec which triggers the garage to close or open just like it would do if you were to hit the button with your hand. 
Here is a picture of the circuit: 

Here is a picture of the server output html page I designed: 
(I know I need to close some windows...)

I would love to post the code on here for the micro controller along with the fritzing files but it is just to much. It is much better to visit my GitHub repository IoTGarage. I must insist that I wrote all of this code, wired the circuit, and fitted plastic storage over the course of about 10 hours so it may be messy and non efficient. Over this project, I learned to use platformIO with atom.io, to program the esp8266 wifi server library, to fit Arduino code onto an orgianl lua board, HTML and CSS from scratch, IFTTT.com, webhooks, dynu static ip setup, and much more.  So there is alot to be learned from building this wifi garage server. The code has more explanation of how everything works, but feel free to ask questions.

A few notes: 
If you would like to access your garage server from outside your house or with google assistant. You should go this route. Set up a free static IP through a service of your choice. I chose https://www.dynu.com/. Then route that traffic through your home router and onto your wifi garage server via port forwarding using a port of your choice. I followed directions HERE which worked out great. It was a little tedious but presents more opportunity to learn. 
Once you have this static outside ip set up, you can then set up a free https://ifttt.com/ account, (if this then that) which is an amazing thing in and of itself. On IFTTT you can set up a web applet that you can connect your google home assistant to your home wifi garage server via webhooks and your all set to go!


When your all finished, you will have a project like this or better: 






Please feel free to leave me some comments or ask and I will answer any questions!
#SudoAptGetUpdateYourBrain!




Monday, April 17, 2017

Spam the Spammers with PYTHON!

Im here today with a little python fun. Below is a short code written in python that, when ran, will send spam email from your gmail inbox to the inbox of your choosing. Happy coding!


import smtplib

def sendemail(from_addr, to_addr_list, cc_addr_list,
              subject, message,
              login, password,
              smtpserver='smtp.gmail.com:587'):
    header  = 'From: %s\n' % from_addr
    header += 'To: %s\n' % ','.join(to_addr_list)
    header += 'Cc: %s\n' % ','.join(cc_addr_list)
    header += 'Subject: %s\n\n' % subject
    message = header + message

    server = smtplib.SMTP(smtpserver)
    server.starttls()
    server.login(login,password)
    problems = server.sendmail(from_addr, to_addr_list, message)
    server.quit()
    return problems

for i in range( times to spam ):
    sendemail(from_addr    = 'yourEmailHere@gmail.com',
              to_addr_list = ['EmailYouWish@toSendTohere'],
              cc_addr_list = [''],
              subject      = 'subject message ',
              message      = 'body'
              login        = 'yourLogin',
              password     = '*********')
    print("message Sent!")

Tuesday, March 28, 2017

Deterministic Finite Automation with Java

Hello everyone, just jumping in to drop off a short little java code demo. For those of computer science, they may enjoy this. For an assignment in Formal Languages and Automata, I was tasked with creating a program that excepted a user input string and then checked each character of the string to see if it is excepted. This machine is known as a DFA, or Deterministic finite automation machine. More on DFA's can be found here: https://en.wikipedia.org/wiki/Deterministic_finite_automaton.
The DFA I was tasked with is figure 2.2 from chapter 2, pg 52 in "An Introduction to Formal Languages and Automata" by Linz, Peter.
Here is the language defining the maching:
Which, as a finite state machine, looks like this: 

I hope you all enjoy the code or it helps your in your understanding and learning. I welcome all comments and questions. Remember, #sudoAptGetUpdateYourBrain
This code can also be found at github here: GettinDatFoShow's DFAdemo (CSsoldier)

Here is the Java Code:  (Edited/Updated to include IOException)

// ********* BEGIN CODE *****************
/*
 *
 */

import java.util.Scanner;
import java.io.IOException;
/**
 *
 * @author Robert Morris
 * This is a program to test out a simple DFA machine. It excepts a input strings
 * from the user and checks them againts the defined language of L={a^n b:n>=0}
 * Since this program is designed to test only strings with 'a' and 'b' in them,
 * (Note: this code has been modified to test any characters the user would like! )
 * it throws an IOException if the user inputs strings containing any other characters.
 */
public class DFAdemo {

    /**
     *
     */
    public static String input;

    public static void main( String[] args ) throws Exception {

        Scanner  scanner = new Scanner( System.in );
        char Char1 = getCharacter(1);
        char Char2 = getCharacter(2);

        message( 1 );
        boolean run = true;

        while( run ){ // loop program

            input = scanner.next().toLowerCase();

            if( input.equals( "0" ) ){ // check to see if user would like to quit test

                run = false;

            }

            else{

                if( dfaMachineTest( input, Char1, Char2) ){ // TEST STRING (TRUE if excepted || FALSE if not excepted)

                    System.out.println( "Thank You." );
                    System.out.println( "Now testing the input: '" +  input + "' ..." );
                    System.out.println( "The string: '" + input + "' was excepted." );

                }

                else{

                    System.out.println( "The string: '" + input + "' was not excepted." );

                }

                message( 2 );
            }
        }

        System.out.println( "Thank you testing this DFA, designed by Rober Morris." );
    }

    public static boolean dfaMachineTest( String inString, char character1, char character2 ) throws IOException, Exception { // machine states test
        /* excepts a user strings, converts it to a character array, checks the characters
            against the definded language and returns true or false if the langauge is excepted */
        Exception IOException = null;
        char[] string = inString.toCharArray();
        int loopLen = string.length;
        char letter;
        int state = -1;

        for( int i = 0; i < loopLen; i++ ){
            letter = string[i];

            // MACHINE STATES BELOW
            if( state == -1 ){ // NO STATE

                if( letter == character1 ){
                    state = 0; // 'a' is encountered
                }

                else if( letter == character2 ){
                    break; //input machine only excepts where the first letter is an character1
                }
                else{
                    throw IOException;// anything other than an character1 or character2 is encountered
                }
            }

            else if(state == 0){ // STATE q0

                if( letter == character2 ){
                    state = 1; // a character2 is encountered
                }

                else if(letter == character1){
                    // remain in state 0
                }

                else{

                    throw IOException;// anything other than an character1 or character2 is encountered
                }

            }

            else if( state == 1 ){ // FINAL STATE q1 ( Excepted Strings End Here)

                if( letter == character1 ){
                    state = 2; // a character1 is encountered
                }

                else if( letter == character2 ){
                    state = 2; // a character2 is encountered
                }

                else{

                    throw IOException;// anything other than an character1 or character2 is encountered
                }

            }

            else if( state == 2 ){ // STATE q2
                // There is no way back to state 1
                if( letter == character1 ){
                    // pass
                }

                else if( letter == character2 ){
                    // pass
                }
                else{
                    throw IOException;// anything other than an character1 or character2 is encountered
                }
            }
        }
        return state == 1; // return true if string is excepted
    }

    public static void message( int message ){
        /* FUNCTION TO DISPLAY MESSAGES TO USER */
        if( message == 1 ){

            String language = "L={a^n b:n>=0}" ;
            System.out.println( "This simple program tests strings against the language defind as: " );
            System.out.println( "            " + language + "          " );
            System.out.println( "Please input a mixed string of 'a' and 'b' to test the DFA, " );
            System.out.println( "Or type '0' to exit: " );incorrect input, only a single character should be typed in!

        }

        else if ( message == 2 ){

            System.out.println( "Please enter another string of 'a's and 'b's to test the DFA," );
            System.out.println( "Or type '0' to exit: " );

        }
        else if ( message == 3 ){

          System.out.println( " Please enter first character for machine:  ");

        }
        else if ( message == 4 ){

          System.out.println( " Please enter second character for machine: ");

        }
    }

    public static char getCharacter(int choice) {
        Scanner  scanner = new Scanner( System.in );
        char result= '#';

        if( choice == 1 ){

            message( 3 );
            String inputChar1 = scanner.next().toLowerCase();
            System.out.println("Thank You.");
            result = inputChar1.charAt(0);

        }

        else if ( choice == 2 ){

            message( 4 );
            String inputChar2 = scanner.next().toLowerCase();
            System.out.println("Thank You.");
            result = inputChar2.charAt(0);

        }

        return result;

    }
}

/// ***************   END CODE ******************


Saturday, February 25, 2017

TCP Client-Server Architecture Simple Demo with Java

TCP Client-Server Architecture Simple Demo, using java.

CSSoldier back again with a new post to help the fellow coders. Today we have some simple java code for a TCP connection. For those that do not know, TCP stands for Transmission Control Protocol. TCP is the most widely used protocol for transferring packets of information across the internet. TCP is a reliable connection protocol that provides ordered and error-checked delivery of information. Needless to say, it is pretty damn important. More information on TCP can be found here: https://en.wikipedia.org/wiki/Transmission_Control_Protocol

This works by way of a Client-Server Architecture. We initiate a point-to-point client-server connection using this Transmission Control Protocol. TCP guarantees the delivery of the data it transmits in the form of "packets". If packets are lost or damaged, TCP will resend the data until it verifies that packets have been successfully transmitted. This is important when you want to make sure your "I love you" message correctly makes it to your loved one....before they think you no longer care and you end up in the dog house. Anyway, moving on...
When establishing connectivity, the client and server each bind a "socket" to their end of the connection. Once a connection has been established, the client and server both read from, and write to, the socket when communicating. Its like picking up the phone, dialing a number and waiting for the connection, when the person picks up, you each listen for the other's communication. You can think of this two way communication with reliability checks like this: As one person talks, the other person will acknowledge what he/she says by saying "ok", "uh huh", "sure", or "I hear ya". If someone cuts out while saying something, you may say something like "I'm sorry, I didn't catch that last bit." Then the person will re-say what they previously said.

Java uses the SeverSocket and Socket classes from the java.net library to accomplish this combination of "IPaddress and port number".

Here is a summary of the Client-Server Architecture:

1. Create a Socket or ServerSocket object and open it over a specified port and IP address or host name.
2. Instantiate InputStreamReader, BufferedReader, and PrintStream objects to stream data to and from each socket. (its like a chaining together of objects)
3. Read from and Write to the stream between the sockets using their agreed-upon protocol.
4. Close the input and output streams.
5. Close the Socket or ServerSocket Objects.

Here is an Illustration:



This code repository can be found on github here: https://github.com/GettinDatFoShow/javaAppLearning.git
Hope you enjoyed today's information and it helps you in your Computer Science or IT learning.
Have a great day and remember, #SudoAptGetUpdateYourBrain

resource used: https://www.youtube.com/watch?v=6G_W54zuadg

Here is each class: (remember that the server socket code must be ran first, obviously)

ServerSock.java:___________________________________________________________

/*
 * This is a Server Socket listening on port 1776 (I call it the freedom port)
 * Once a connection is heard from a client, a socket opened and accepted.
 * from the socket instance that is created, there are a few properties available.
 * The .getInetAdress() from the socket returns an InetAddress instance of the client on the
 * other side of the socket. we can then call the .getHostAddress on the InetAddress object
 * and obtain a string of the clients IPaddress.
 * Then, an InputStreamReader object is created using the socket as its parameter.
 * After this, a BufferedReader object is created using the InputStreamReader object as the parameter.
 * The socket is now ready listen for messages sent through the socket from the client.
 * there is a while loop once the readline() function is called appon the bufferedReader object
 * becuase the readline() will stop at the \n character. If it doesnt keep looping it will not be
 * able to read out multiple lines from the client.
 * Once the client connects the server sends a message back with its connecting IPaddress,
 * alonge with the date. Then sends and empty string (which will have a '/n' character appended,
 * so that the client will then close the socket (this is purely for demo purposes and is may not be needed).
*/
package tcpdemo;
import java.io.*;
import java.net.*;
import java.util.Date;
/**
 *
 * @author Robert Morris
 */
public class ServerSock {
 
    public static void main(String[] args) throws Exception{
     
        ServerSock Server = new ServerSock();
        Server.run();
     
    }
 
 
    public void run() throws Exception{
     
        String messageIn = null;
     
        ServerSocket serverSocket = new ServerSocket(1776);
        Socket socket = serverSocket.accept();
        // Information Regarding the Connection.
        InetAddress INA = socket.getInetAddress();
        String hostIP = INA.getHostAddress();
        Date date = new Date();
        InputStreamReader inRead = new InputStreamReader(socket.getInputStream());
        BufferedReader bRead = new BufferedReader(inRead);
     
        while (( messageIn = bRead.readLine()) != null){
            if (messageIn.length() <= 2){
                break;
            }
            System.out.println(messageIn);
            String messageOut = "Your IP address is: "+ hostIP + "\n"
                    + "It is now: " + date;
            PrintStream pStream = new PrintStream(socket.getOutputStream());
            pStream.println(messageOut);
            pStream.println("");
            break;
         
        }
        socket.close();
    }
 
}

// ***** end SeverSock.java *****

_________________________________________________________________________

ClientSock.java:___________________________________________________________

/*
 *This is a client socket class that connects to the server socket
 * on a pre-specified of 1776, supplying a name of 'localhost'
 * once the socket is accepted by the server a PrintStream object is created with
 * the socket.getOutputStream() as a parameter, following this creation, a message is sent
 * to the server. 
 * Then, an InputStreamReader object is created using the socket as its parameter.
 * After this, a BufferedReader object is created using the InputStreamReader object as the parameter.
 * The socket is now ready listen for messages sent through the socket from the client.
 * there is a while loop once the readline() function is called appon the bufferedReader object
 * becuase the readline() will stop at the \n character. If it doesnt keep looping it will not be
 * able to read out multiple lines from the server. 
 * a while loop is created for the bufferedReader to be able to read multiple lines so that
 * it does not terminate until a specified message is received. In this case the specified message is 
 * a 'new line' character. 
 */
package tcpdemo;

import java.io.*;
import java.net.*;

/**
 *
 * @author Robert Morris
 */// ***** end ClientSock.java *****// ***** end ClientSock.java *****
public class ClientSock {
    
   public static void main(String [] args) throws Exception{
       
       ClientSock client = new ClientSock();
       client.run();
       
   } 
    
   public void run() throws Exception{
       String messageIn = null;
       Socket socket = new Socket("localhost", 1776);
       PrintStream pStream = new PrintStream(socket.getOutputStream());
       pStream.println("I NEED TO CONNECT NOWWWWW!!!!");
       
       InputStreamReader inRead = new InputStreamReader(socket.getInputStream());
       BufferedReader bRead = new BufferedReader(inRead);
       
       
       while((messageIn = bRead.readLine()) != null){
            
           if (messageIn.length() <= 2){
                break;
            }
              System.out.println(messageIn);
       }
       socket.close();
              
   }
    
}

// ***** end ClientSock.java *****

Friday, February 17, 2017

Data Analytics: MLE proof as biased estimator of Variance

More Data Analytics here, Here is a proof for the Variance Maximum Likelihood Estimator as a biased estimator of Variance.


Hopefully I did everything right there. 
I also wrote matlab code that compares the Sample Variance, which is shown here:
to the Maximum Likelihood Estimator for Variance. 
Here is what the code output will look like :


And here, I present you with my matlab code which can all be found at: https://github.com/GettinDatFoShow/MatLab_CODE

%% __________________________BEGIN MLE MATLAB CODE ________________________
%% ------------------------------------------------------------------------------------------------------------------

% Variance Maximum likelihood estimator tests
% @author Robert Morris - Delaware State University
% Data Analytics - Project 2 
% May take 5 - 10 seconds or so to run. 


% This Project Starts With a randomly generated 
% Population of weights for 100,000 individuals. 
% It then caluculates the mean and variance of the weights
% Each Graph/plot is explained, please Maximize figure after running for 
% Full effect.

clc; clear; close all

numPopulation= 1000000;
maxWeight= 300;
minWeight= 100;
populationWeights= zeros(1,numPopulation);



% randomly create population of ages


for i=1:1:numPopulation
    populationWeights(1,i) = randi([minWeight,maxWeight], 1);
end

meanWeight= mean(populationWeights); % find trandomly generated 
% Population of weights for 100,000 individuals. 
% It then caluculates the mean and variance of the weights
% In the overall population.
% Then it conducts a test of 10 different previously selected 
% sample sizes from the population. For each sample size, a 1000
% trials are conducted in which the program calculates the sample size 
% mean and variance, recording eahistogram(MLEVarsHisto, bins, 'facecolor', 'm');ch value during each trial.
% at the end of each trial the overall sample variance mean, sample 
% variance max outlier, and sample variance min outlier of the trials are recorded.
% After the trials are ran for each previously selected sample size, the results are 
% displayed in a figure for comparrison. The plot in the figure shows how the 
% sample variance numbers converge toward the overall population weight variance 
% as the sample size grows. he population mean weight.
weight= 0;
xtickangle(45);
for i=1:1:numPopulation
    weight= weight + (populationWeights(1,i) - meanWeight)^2;
end

weightVar= weight/numPopulation; % notice original variance calculation of size n


experiments=10;
sampleSize=100;
sampleXaxis = zeros(1,experiments);
actualWeightVars = zeros(1, experiments);
MLEVars = zeros(1, experiments);
sampleVars = zeros(1, experiments);
samples = zeros(1,sampleSize);
sampMeanWeight = 0;
sampleVariance = 0;
MLEVariance = 0;

for j=1:1:experiments % sample mean Finder
   
   for k=1:1:sampleSize
     whichPerson= randi(numPopulation-1,1) + 1; 
     samples(1,k) = populationWeights(1,whichPerson);
   end

   sampleMean= mean(samples);  % find the sample mean
   sampleWeight= 0;

   for l=1:1:sampleSize % sample Variance Finder
      sampleWeight = sampleWeight + (samples(1,l)-sampleMean)^2;
   end

   sampleVariance = (1/(sampleSize-1)) * sampleWeight; % sample variance calculation of size n-1
   MLEVariance = (1/(sampleSize)) * sampleWeight; % max likelihood sample calculation of size n
   SampleVars(1,j) = sampleVariance;
   MLEVars(1, j) = MLEVariance;
   actualWeightVars(1, j) = weightVar;
   sampleXaxis(1,j) = sampleSize;
   sampleSize = sampleSize * 2;
end   
   
fig = figure;
set(0, 'defaultfigureposition', [1300 10 900 600])
fig.NumberTitle = 'off';
fig.Name = 'Variance Maximum Likelihood Test';
hold on;

subplot(2,2,1);
x = linspace(1,10,10);
plot(x, SampleVars, 'b', x, MLEVars, 'g', x, actualWeightVars, 'r--', 'lineWidth', 2);
xticks([1,2,3,4,5,6,7,8,9,10]);
xticklabels({'100','200','400','800','1600','3200','6400','12800','25600', '51200'});
title('Variance Estimators Comparison');
xlabel('Variance Experiment Sample Sizes');
ylabel('Calculated Variance For Weights');
xtickangle(45);
legend('show');
legend({'Sample Variance', 'Variance MLE', 'Population Variance'});
lcn = 'northeast';

testMLEVarianceMax = zeros(1,experiments);
testMLEVarianceMin = zeros(1,experiments);
testMLEVarianceMean = zeros(1,experiments);
actualWeightVars = zeros(1, experiments);
sampleSize=10;

for n=1:1:experiments
    
   samples= zeros(1,sampleSize);
   numTrials=500;
   sampMeanWeight = 0;xtickangle(45);
   MLEVars = zeros(1, numTrials);
   MLEVarMax = 0;
   MLEVarMin = 0;
   MLEMean = 0;
   
   for j=1:1:numTrials % sample mean Finder

       for k=1:1:sampleSize
         whichPerson= randi(numPopulation-1,1) + 1; 
         samples(1,k) = populationWeights(1,whichPerson);
       end

       sampleMean= mean(samples);  % find the sample mean
       sampleWeight= 0;

       for l=1:1:sampleSize % sample Variance Finder
          sampleWeight = sampleWeight + (samples(1,l)-sampleMean)^2;
       end

       MLEVariance = (1/(sampleSize)) * sampleWeight; % sample variance calculation of size n-1 
       MLEVars(1,j) = MLEVariance;

   end
   
   MLEVarMax = max( MLEVars ); % max outlier of the sample variance
   MLEVarMin = min( MLEVars ); % min outlier of the sample variance
   MLEMean = mean( MLEVars ); % the average sample variance of the samples
   
   testMLEVarianceMax(1,n) = MLEVarMax;
   testMLEVarianceMin(1,n) = MLEVarMin;
   testMLEVarianceMean(1,n) = MLEMean;
   actualWeightVars(1,n) = weightVar;
   sampleSize = sampleSize*2;
   
end

subplot(2,2,2);
x = linspace(1,10,10);
plot(x, testMLEVarianceMax, 'b--', x, testMLEVarianceMin, 'g--', x, testMLEVarianceMean, 'c--*', x, actualWeightVars, 'r--', 'LineWidth',2);
xticks([1,2,3,4,5,6,7,8,9,10]);
xticklabels({'10','20','40','80','160','320','640','1280','2560', '5120'});
title('MLE Variance Convergence');
xlabel('Variance Sample Sizes');
ylabel('Calculated Variance For Weights');
xtickangle(45);
legend('show');
legend({'MLE Var Max', 'MLE Var Min', 'MLE Var Mean', 'Population Variance'});
lcn = 'northeast';


histoTestSize = 1000;
MLEVarsHisto = zeros(1, histoTestSize);
sampleSize = 5000;
samples = zeros(1,sampleSize);

for t=1:1:histoTestSize

   for p=1:1:sampleSize
     whichPerson= randi(numPopulation-1,1) + 1; 
     samples(1,p) = populationWeights(1,whichPerson);
   end

   sampleMean= mean(samples);  % find the sample mean
   sampleWeight= 0;

   for m=1:1:sampleSize % MLE Variance Helper
      sampleWeight = sampleWeight + (samples(1,m)-sampleMean)^2;
   end

   MLEVariance = (1/(sampleSize)) * sampleWeight; % max likelihood sample calculation of size n
   MLEVarsHisto(1, t) = MLEVariance;

end



SampleVarsHisto2 = zeros(1, histoTestSize);
sampleSize2 = 5000;
samples2 = zeros(1,sampleSize2);
sampleVariance2 = 0;

for t=1:1:histoTestSize

   for p=1:1:sampleSize2
     whichPerson= randi(numPopulation-1,1) + 1; 
     samples2(1,p) = populationWeights(1,whichPerson);
   end

   sampleMean= mean(samples2);  % find the sample mean
   sampleWeight= 0;

   for m=1:1:sampleSize2 % MLE Variance Helper
      sampleWeight = sampleWeight + (samples2(1,m)-sampleMean)^2;
   end

   sampleVariance2 = (1/(sampleSize2-1)) * sampleWeight; % max likelihood sample calculation of size n
   SampleVarsHisto2(1, t) = sampleVariance2;

end

    histoEffect = zeros(1,80);
    histoEffect2 = zeros(1,80);

for v = 1:1:80

    histoEffect(1, v) = weightVar;

end

bins = 50;
subplot(2,2,3);
hold on;
histogram(MLEVarsHisto, bins, 'facecolor', 'm');
title('MLE Variance Distribution, 5k Sample Size Test ');
xlabel('MLE Variances ');
ylabel('Totals Calculated');
xtickangle(45);
histogram(histoEffect, bins, 'facecolor', 'r', 'BinWidth', 2);

subplot(2,2,4);
hold on;
histogram(SampleVarsHisto2, bins, 'facecolor', 'g');
title('Sample Variance Distribution, 5k Sample Size Test ');
xlabel('Sample Variances ');
ylabel('Totals Calculated');
xtickangle(45);
histogram(histoEffect, bins, 'facecolor', 'r', 'BinWidth', 2);

%% ------------------------------------------------------------------------------------------------------------------
%% ____________________________ END MLE MATLAB CODE _______________________

Saturday, February 4, 2017

Data Analytics: Proof s² is an unbiased estimator of Variance or σ2

Here is a little data analytics for you guys. Here is a mathematical proof that  Sample Variance, otherwise written as , is an unbiased estimator of Variance or σ2. 




And Here Is a Matlab Code that make this consistency apparent as sample sizes of the main population grow larger. 

===============START MATLAB CODE======================

% estimatorSampleVarianceConsistencyTest
% @author Robert Morris - Delaware State University
% Data Analytics - Project 2


% This Project Starts With a randomly generated
% Population of weights for 100,000 individuals.
% It then caluculates the mean and variance of the weights
% In the overall population.
% Then it conducts a test of 10 different previously selected
% sample sizes from the population. For each sample size, a 1000
% trials are conducted in which the program calculates the sample size
% mean and variance, recording each value during each trial.
% at the end of each trial the overall sample variance mean, sample
% variance max outlier, and sample variance min outlier of the trials are recorded.
% After the trials are ran for each previously selected sample size, the results are
% displayed in a figure for comparrison. The plot in the figure shows how the
% sample variance numbers converge toward the overall population weight variance
% as the sample size grows.

clc; clear; close all

numPopulation= 100000;
maxWeight= 300;
minWeight= 100;
populationWeights= zeros(1,numPopulation);



% randomly create population of ages


for i=1:1:numPopulation
    populationWeights(1,i) = randi([minWeight,maxWeight], 1);
end

meanWeight= mean(populationWeights); % find trandomly generated
% Population of weights for 100,000 individuals.
% It then caluculates the mean and variance of the weights
% In the overall population.
% Then it conducts a test of 10 different previously selected
% sample sizes from the population. For each sample size, a 1000
% trials are conducted in which the program calculates the sample size
% mean and variance, recording each value during each trial.
% at the end of each trial the overall sample variance mean, sample
% variance max outlier, and sample variance min outlier of the trials are recorded.
% After the trials are ran for each previously selected sample size, the results are
% displayed in a figure for comparrison. The plot in the figure shows how the
% sample variance numbers converge toward the overall population weight variance
% as the sample size grows. he population mean weight.
weight= 0;

for i=1:1:numPopulation
    weight= weight + (populationWeights(1,i) - meanWeight)^2;
end

weightVar= weight/numPopulation; % notice original variance calculation of size n
   
sampleSize=10;

% Test Sample Variance Varibles to show
% Convergence apon larger sample sizes
experiments=10;
sampleXaxis = zeros(1,experiments);
testSampleVarianceMax = zeros(1,experiments);
testSampleVarianceMin = zeros(1,experiments);
testSampleVarianceMean = zeros(1,experiments);
actualWeightVars = zeros(1, experiments);

for n=1:1:experiments
   
   samples= zeros(1,sampleSize);
   numTrials=100;
   sampMeanWeight = 0;
   sampleVars = zeros(1, numTrials);
   sampleVarMax = 0;
   sampleVarMin = 0;
   sampleVarMean = 0;
  
   for j=1:1:numTrials % sample mean Finder

       for k=1:1:sampleSize
         whichPerson= randi(numPopulation-1,1) + 1;
         samples(1,k) = populationWeights(1,whichPerson);
       end

       sampleMean= mean(samples);  % find the sample mean
       sampleWeight= 0;

       for l=1:1:sampleSize % sample Variance Finder
          sampleWeight = sampleWeight + (samples(1,l)-sampleMean)^2;
       end

       sampleVariance = (1/(sampleSize-1)) * sampleWeight; % sample variance calculation of size n-1
       sampleVars(1,j) = sampleVariance;

   end
  
   sampleVarMax = max( sampleVars ); % max outlier of the sample variance
   sampleVarMin = min( sampleVars ); % min outlier of the sample variance
   sampleVarMean = mean( sampleVars ); % the average sample variance of the samples
  
   testSampleVarianceMax(1,n) = sampleVarMax;
   testSampleVarianceMin(1,n) = sampleVarMin;
   testSampleVarianceMean(1,n) = sampleVarMean;
   actualWeightVars(1,n) = weightVar;
   sampleXaxis(1,n) = sampleSize;
   sampleSize = sampleSize*2;
  
end

fig = figure;
set(0, 'defaultfigureposition', [1300 10 900 600])
fig.NumberTitle = 'off';
fig.Name = 'Sample Variance Consistency Experment From Population Weights';

x = linspace(1,10,10);
plot(x, testSampleVarianceMax, 'b--', x, testSampleVarianceMin, 'g--', x, testSampleVarianceMean, 'c--*', x, actualWeightVars, 'r--', 'LineWidth',2);
xticks([1,2,3,4,5,6,7,8,9,10]);
xticklabels({'10','20','40','80','160','320','640','1280','2560', '5120'});
title('Sample Variation to Actual Variation Convergence');
xlabel('Variance Experiment Sample Sizes');
ylabel('Calculated Variance For Weights');
xtickangle(45);
legend('show');
legend({'Samp Var Max', 'Samp Var Min', 'Samp Var Mean', 'Pop Weight Variance'});
lcn = 'northeast';

===============END MATLAB CODE=========================




Here is a shot what the code produces: 




Help this helps everyone on their path to becoming better computer scientist. 
Enjoy and #codeON #sudoAptGetUpdateYourBrain