CTL-Reports

Nicolas A. Baginsky

Report 6/10

intermediate report, goals, assumptions and thoughts about
the preparations for my final experiment
in the "closing the loop" lab at "time's up" in Linz

-by analysing what had happened during the initial experiment at ctl-lab (see text by jam, scheme by nab) we figured we would need a non human entity that should be able to act as a objective observer in coming experiments. since artificial neural networks are my favourite tools for classification of complex real-world data, i suggested to use a Tuevo Kohonen network to fulfil this task. these Kohonen networks, also called SOMs (Self Organising Maps), have the property to classify incoming data without any pre-defined knowledge about the meaning of this data. they are able to generate categories for classification purely from the presented data and the qualities hidden within.

we were then discussing the various possibilities of how to engage such a neural network into the different types of experiments that will be made at ctl-lab this year. what kind of input should be used? what information will probably be available in all the different set-ups yet to come? here is what we came up with:
we will use a Kohonen Network with thirteen input dimensions and two output dimensions.
these are the inputs we are going to use:
- skin resistance measured at left hand pointing finger.
- muscle tonus of right hand middle finger measured at the inside of right fore-arm.
- seven channel fft output of brain wave potentials measured with three electrodes on forehead and processed by an IBVA device and software.
- four channel "the eye" data. "the eye" is a device that measures brightness at four clever chosen locations on a video monitor. this is done with photo resistors attached to the screen. their values are digitised by a basic-stamp and send via serial link to the Kohonen network.

the two-dimensional output layer should have a maximum of 128*128=16384 neurons. that way the classification result of the neural network could for example be interpreted as two-channel midi data (x an y position of the "winner" neuron = note-on, "winner" accuracy = velocity).
in a later discussion we agreed not to immediately apply the network output to the experiment because this would destroy the network's objectivity. coupling the network results to a sound- or other output-device would have an influence on the proband and therefore close the loop and engage the neuronet into the experiment.
still we are going to save the network output together with all the sensor data for later inspection and analysis. another tool for evaluating the experiments or rather the differences between the experimental process with different probands will be the recordings of the graphical output of the networks. for every session the neuronet will be initialised at random. we will choose three of the thirteen input dimensions and display them as a three-dimensional grid that will change it's perspectively drawn shape according to the network development. this graphical output will be saved on every learning iteration and later be assembled into quicktime movies. by watching the movies generated by the different test persons we should be able to notice significant differences and to compare the sessions on a very abstract level.

NAB, 06/10/98




	Nicolas A. Baginsky Report 6/10	intermediate report, goals, assumptions and thoughts about the preparations for my final experiment in the "closing the loop" lab at "time's up" in Linz -by analysing what had happened during the initial experiment at ctl-lab (see text by jam, scheme by nab) we figured we would need a non human entity that should be able to act as a objective observer in coming experiments. since artificial neural networks are my favourite tools for classification of complex real-world data, i suggested to use a Tuevo Kohonen network to fulfil this task. these Kohonen networks, also called SOMs (Self Organising Maps), have the property to classify incoming data without any pre-defined knowledge about the meaning of this data. they are able to generate categories for classification purely from the presented data and the qualities hidden within. we were then discussing the various possibilities of how to engage such a neural network into the different types of experiments that will be made at ctl-lab this year. what kind of input should be used? what information will probably be available in all the different set-ups yet to come? here is what we came up with: we will use a Kohonen Network with thirteen input dimensions and two output dimensions. these are the inputs we are going to use: - skin resistance measured at left hand pointing finger. - muscle tonus of right hand middle finger measured at the inside of right fore-arm. - seven channel fft output of brain wave potentials measured with three electrodes on forehead and processed by an IBVA device and software. - four channel "the eye" data. "the eye" is a device that measures brightness at four clever chosen locations on a video monitor. this is done with photo resistors attached to the screen. their values are digitised by a basic-stamp and send via serial link to the Kohonen network. the two-dimensional output layer should have a maximum of 128*128=16384 neurons. that way the classification result of the neural network could for example be interpreted as two-channel midi data (x an y position of the "winner" neuron = note-on, "winner" accuracy = velocity). in a later discussion we agreed not to immediately apply the network output to the experiment because this would destroy the network's objectivity. coupling the network results to a sound- or other output-device would have an influence on the proband and therefore close the loop and engage the neuronet into the experiment. still we are going to save the network output together with all the sensor data for later inspection and analysis. another tool for evaluating the experiments or rather the differences between the experimental process with different probands will be the recordings of the graphical output of the networks. for every session the neuronet will be initialised at random. we will choose three of the thirteen input dimensions and display them as a three-dimensional grid that will change it's perspectively drawn shape according to the network development. this graphical output will be saved on every learning iteration and later be assembled into quicktime movies. by watching the movies generated by the different test persons we should be able to notice significant differences and to compare the sessions on a very abstract level. NAB, 06/10/98