Mini-project: Histograms R pretty cool - uni-dimensional binary classification example

A few months ago I had to solve a pretty interesting, though not very difficult, assignment for my Machine Learning class.  I really think that this is the very first lecture in any Machine Learning Class around the world.
.

So, given a data set containing 4 distinct features and 2 possible outcomes (or classes, noted for simplicity with -1 and +1) the task was to determine if any of the 4 features would be suitable for the construction of a uni-dimensional classifier. In plain English, determining if by considering just one single feature one could accurately predict if a new data sample is a "-1" or a "+1".

.

Apparently a very nice way to check the individual quality of such features is to plot a joint histogram of the two given output classes (marked by +1 and -1). And plotting the histogram of any characteristic with regards to a given domain is a rather simple thing: divide the domain in a certain number of equal sub-intervals and for each sub-interval just count the number of records that have the characteristic you are interested in. In our case the domain is actually the interval between the smallest and highest value of the given feature and the characteristic is the output class we are interested in.

.

The idea was to plot the joint histogram of the two classes for each feature and see if there is a threshold that could separate them.

.

So here is how the input file is structured. The file starts with a header line that contains the name of the 4 fields containing features, F1 to F4 in our case and the name that marks the outcome field, R. On each of the next data lines, we find 4 real values (in the interval [0,1]), one for each feature and the outcome class. For example, the beginning of the file might look like this:

F1   ,F2   ,F3   ,F4   , R
0.275,0.975,0.304,0.638,-1
0.665,0.240,0.241,0.804,-1
0.129,0.388,0.754,0.717,-1
0.832,0.368,0.988,0.271, 1
0.956,0.820,0.787,0.012, 1

And here is what the output should look like for a file containing 100 data lines(green for -1 class, red for +1 class, brown for overlapping, and the blue line marks the optimal threshold value found by means of brute force):

Obviously, for the given input, the only feature that seems suitable for uni-dimensional classification is Feature nr. 1.

.

You can download a script I have written in R that outputs such joint histograms from well formatted input files by clicking here or from the Downloads box (JoinHistogramsInR.zip).

Application platform: Platform independent

Mini-project: Basic Systolic Algorithms

Here are two very basic simulation applications for two systolic algorithms (aka. Instruction Systolic Arrays). Their main purpose is to illustrate how such systolic algorithms work on a given data set. Both simulators have been designed for the Systolic Algorithms course I've attended at my university. 

Application no.1: Systolic Array Sorting 

The first algorithm I have implemented aims to solve the simple problem of Systolic Array Sorting. The idea of the problem is the following: given an uni-dimensional systolic system large enough to store the entire contents of an arbitrary array, devise a method of sorting the array in linear time (i.e. algorithm complexity of O(n)).

The systolic system that solves this problem is extremely simple:

1.Each systolic cell can store at most two values (a base value and an auxiliary transport value).
2.Each cell communicates with its left and right neighbors (it can receive a value from the left neighbor and can pass a value to its right neighbor = data flow model). When a cell receives a value from the left it stores it as a transport value.
3.At each step (system tact) each systolic cell (that holds both a base and a transport value) takes a simple decision: it passes to its left neighbor the lowest value between the base value and the transport value and stores the highest of the two.If the cell holds only a base value then it does not take any action.

The systolic system is initialized by loading the initial unsorted array (each array element is loaded as the base value of a cell cell). The output of the system (i.e. the right-most cell) should be connected to a data structure where we wish to store the sorted array. 

.

At each step we simply input into the systolic system a predefined maximal value (i.e. a value greater than any value in the unsorted array) by "communicating" it to the left-most cell of the systolic system.

.

The entire key (and idea behind any systolic algorithm) is that the "communication" process is parallel.

. 

If we have an array of size n the sorting process will take exactly 2*n steps which translates into a complexity of O(n). 

Application no.2: Systolic Matrix Multiplication

The second application simulates systolic matrix multiplication. The systolic architecture needed to solve this problem is a bit more complicated than the one used in the first example. The cells are organized as a matrix (the size of which is equal to the one of the expected result matrix). Each cell can communicate with its 4 neighbors (receives values from the cells to the left and above and sends them to the cells to the right and below). Also, the individual organization of each cell is more complicated, as each cell can hold 3 values and execute 2 operations (addition and multiplication). For a detailed explanation of the systolic matrix multiplication algorithm please see this page.

.

The complexity of this systolic algorithm is O(n) as it would take 3*n steps to multiply two matrices of size nxn. In order to simplify the observation of the process, in the simulation application, each step is further divided into 3 sub-operations.

Although fairly simple, these two examples can easily show the power of systolic architectures as they offer linear solutions to problems which normally have a higher complexity (O(n*log(n)) for sorting and O(n^2.376) for matrix multiplication)).

.

You can download these two simulators by clicking here or from the Downloads box (SystolicSimulators.zip).  

Application platform: WinNT

Mini-project: DCrypt

DCrypt is a small program I wrote for my network security class. It features 2 encryption ciphers and the DES algorithm. It was written in Delphi and allows for text to hexadecimal encryption and hexadecimal to text decryption.
.

The first cipher I included in DCrypt is the Viginere cipher. You can find out detailed information about this encryption method here. In this implementation, the cipher can process full ASCII text but only encodes the letters (A-Z). It also converts small letters to capital letters.

.

The second cipher I implemented is the Bifid cipher. In the current implementation this cipher can only process text containing: A-Z letters and a few special characters. It too converts small letters to capital leters.

.

Finally, the DES implementation encodes full ASCII text using 64 bit blocks and a 64 bit key and does not perform any conversions on the text. The code for this part of the application is largely based on a Delphi unit written by Francoise PIETTE.

.

You cand download a fully working version of the application (source code included) by clicking here or from the Downloads box (DCrypt.zip). Here are two screenshots:

Application platform: WinNT

Mini-Project: Shutdown Timer

Lately I got used to falling asleep while watching TV on my laptop and as my TV-tuner software is quite primitive and doesn't have the option to shutdown the computer after a predefined period of time, I started to search the web for a simple shutdown timer application. I didn't want a fancy application with a lot of options just a basic program that will enable me to select a period of time after which the system would shutdown and that, if needed, could cancel the shutdown command.
.
As the first two applications found by Google that met my over demanding request were not free I decided to write my own program. I remembered a nice and useful piece of code I once stumbled upon on Torry's Delphi Pages that enables a process to aquire privileges on WinNT platforms. I used it to aquire the 'SeShutdownPrivilege' and after that, the rest of the program was done in no more than 15 minutes.
.
You cand download a fully working version of Shutdown Timer (source code included) by clicking here or from the Downloads box (ShutdownTimer.zip). Ohh ... and here is a screenshot of my latest mesmerizing achievement in the field of computer software:

Application platform: WinNT

Edit: Application now supports one command line parameter: a numeric value indicating the number of minutes till shutdown.