Commit b2558889 authored by Jussi Lindgren's avatar Jussi Lindgren

Plugins: Removed StimulusSender code from the Graz Visualization box

- Stimulations should now be sent to AS after rendering by the box
- Wrote the missing box documentation
- Minor cleanup
parent 95eea1d2
......@@ -6,6 +6,20 @@ Detailed description
__________________________________________________________________
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Description|
The Graz visualization box implements the display of the well-known
Graz BCI paradigm, typically used for Motor Imagery BCI experiments.
The paradigm is described e.g. in Pfurtscheller & Neuper,
"Motor Imagery and Direct Brain-Computer Communication", 2001.
In its most common use, the Graz visualization instructs the user to
perform either left or right hand motor imagery by presenting left and
right arrows, correspondingly. In order to know when to present what,
the box must be driven with a stimulation stream that provides the
timeline of the experiment, including trial starts, trial stops, etc.
The same box can be used both for data collection and for real-time
displays. When used real time, the box can display a directional blue
bar to illustrate the strength and direction of the prediction.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Description|
__________________________________________________________________
......@@ -16,10 +30,14 @@ __________________________________________________________________
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Inputs|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Input1|
The timeline of the events.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Input1|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Input2|
For online use and feedback, the strength of the current activation.
This can be for example the continuous output from a classifier.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Input2|
__________________________________________________________________
Settings description
......@@ -29,10 +47,29 @@ __________________________________________________________________
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Settings|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting1|
If true, the user will be shown the arrows.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting1|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting2|
If true, the current activation will be presented as feedback in form of a blue bar.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting2|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting3|
If true, feedback will be shown only after the trial. Otherwise immediately.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting3|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting4|
If true, a little matrix will display how many online trials matched the arrow direction.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting4|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting5|
How many predictions to integrate for computing the feedback bar?
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting5|
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Setting6|
If true, the blue bar will only be displayed if its direction agrees with the arrow.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Setting6|
__________________________________________________________________
Online visualisation settings
......@@ -54,5 +91,15 @@ Miscellaneous description
__________________________________________________________________
* |OVP_DocBegin_BoxAlgorithm_GrazVisualization_Miscellaneous|
The timeline required by the box can be generated by a Lua stimulator.
OpenViBE is bundled with a few motor imagery examples illustrating
this (in folder "bci-examples/").
In order to place the markers (stimulations) to the recorded EEG stream accurately in time,
the box connects to the Acquisition Server's TCP Tagging plugin and forwards the received
timeline there after rendering. The subsequent scenarios and writers should then use the timeline
from the Acquisition Server output and not directly from the timeline generating box. But
due to the long duration of time the motor imagery is typically integrated, this paradigm
could be less sensitive to marker alignment issues compared e.g. to P300.
* |OVP_DocEnd_BoxAlgorithm_GrazVisualization_Miscellaneous|
*/
// @todo for clarity, the StimulusSender related code blocks should be pushed inside the class and away from here
#include "ovpCBoxAlgorithmP300SpellerVisualisation.h"
#include <system/ovCMemory.h>
......
// @todo for clarity, the StimulusSender related code blocks should be pushed inside the class and away from here
#include "ovpCGrazVisualization.h"
#include <cmath>
......@@ -24,92 +27,9 @@ using namespace std;
#include <fstream>
#include <cstdlib>
#include <boost/array.hpp>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <sys/timeb.h>
using boost::asio::ip::tcp;
/*
* \class StimulusSender
* \brief Basic illustratation of how to read from TCPWriter using boost::asio
*/
class StimulusSender {
public:
~StimulusSender()
{
if(m_oStimulusSocket.is_open())
{
std::cout << "Disconnecting\n";
m_oStimulusSocket.close();
}
}
StimulusSender(void)
: m_oStimulusSocket(m_ioService), m_bConnectedOnce(false)
{
}
boolean connect(const char* sAddress, const char* sStimulusPort)
{
//m_pStimulusSocket = new tcp::socket(m_ioService);
boost::system::error_code error;
tcp::resolver resolver(m_ioService);
// Stimulus port
std::cout << "Connecting to stimulus port [" << sAddress << " : " << sStimulusPort << "]\n";
tcp::resolver::query query = tcp::resolver::query(tcp::v4(), sAddress, sStimulusPort);
m_oStimulusSocket.connect(*resolver.resolve(query), error);
if(error)
{
std::cout << "Connection error: " << error << "\n";
return false;
}
m_bConnectedOnce = true;
return true;
}
boolean sendStimuli(uint64 ui64Stimuli)
{
if(!m_bConnectedOnce) {
return false;
}
timeb time_buffer;
ftime(&time_buffer);
if(!m_oStimulusSocket.is_open())
{
std::cout << "Cannot send stimulation, socket is not open\n";
return false;
}
uint64 l_ui64tmp = 0;
try
{
boost::asio::write(m_oStimulusSocket, boost::asio::buffer((void *)&l_ui64tmp, sizeof(uint64)));
boost::asio::write(m_oStimulusSocket, boost::asio::buffer((void *)&ui64Stimuli, sizeof(uint64)));
boost::asio::write(m_oStimulusSocket, boost::asio::buffer((void *)&l_ui64tmp, sizeof(uint64)));
}
catch (boost::system::system_error l_oError)
{
std::cout << "Issue '" << l_oError.code().message().c_str() << "' with writing stimulus to server\n";
}
return true;
}
private:
boost::asio::io_service m_ioService;
tcp::socket m_oStimulusSocket;
OpenViBE::boolean m_bConnectedOnce;
};
#include "../ovpCStimulusSender.h"
//////////////////////////
......@@ -117,6 +37,15 @@ namespace OpenViBEPlugins
{
namespace SimpleVisualisation
{
// This callback flushes all accumulated stimulations to the TCP Tagging
// after the rendering has completed.
gboolean flush_callback(gpointer pUserData)
{
((CGrazVisualization*)pUserData)->flushQueue();
return false; // Only run once
}
gboolean GrazVisualization_SizeAllocateCallback(GtkWidget *widget, GtkAllocation *allocation, gpointer data)
{
reinterpret_cast<CGrazVisualization*>(data)->resize((uint32)allocation->width, (uint32)allocation->height);
......@@ -152,7 +81,6 @@ namespace OpenViBEPlugins
m_f64BarScale = l_f64Prediction;
}
break;
m_pStimulusSender->sendStimuli(m_ui64LastStimulation);
case OVTK_GDF_End_Of_Session:
m_eCurrentState = EGrazVisualizationState_Idle;
......@@ -163,7 +91,6 @@ namespace OpenViBEPlugins
drawBar();
}
break;
m_pStimulusSender->sendStimuli(m_ui64LastStimulation);
case OVTK_GDF_Cross_On_Screen:
m_eCurrentState = EGrazVisualizationState_Reference;
......@@ -223,6 +150,12 @@ namespace OpenViBEPlugins
{
processState();
}
// Queue the stimulation to be sent to TCP Tagging
{
boost::mutex::scoped_lock lock(m_oIdleFuncMutex);
m_vStimuliQueue.push_back(m_ui64LastStimulation);
}
}
void CGrazVisualization::processState()
......@@ -318,6 +251,11 @@ namespace OpenViBEPlugins
m_i64PredictionsToIntegrate = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 4);
m_bPositiveFeedbackOnly = FSettingValueAutoCast(*this->getBoxAlgorithmContext(), 5);
m_pStimulusSender = NULL;
m_uiIdleFuncTag = 0;
m_vStimuliQueue.clear();
if(m_i64PredictionsToIntegrate<1)
{
this->getLogManager() << LogLevel_Error << "Number of predictions to integrate must be at least 1!";
......@@ -395,7 +333,7 @@ namespace OpenViBEPlugins
if(!m_pStimulusSender->connect("localhost", "15361"))
{
this->getLogManager() << LogLevel_Warning << "Unable to connect to AS TCP Tagging, stimuli wont be forwarded.\n";
this->getLogManager() << LogLevel_Warning << "Unable to connect to AS's TCP Tagging plugin, stimuli wont be forwarded.\n";
}
return true;
......@@ -403,6 +341,16 @@ namespace OpenViBEPlugins
boolean CGrazVisualization::uninitialize()
{
{
boost::mutex::scoped_lock lock(m_oIdleFuncMutex);
if(m_uiIdleFuncTag)
{
m_vStimuliQueue.clear();
g_source_remove(m_uiIdleFuncTag);
m_uiIdleFuncTag = 0;
}
}
if(m_pStimulusSender)
{
delete m_pStimulusSender;
......@@ -453,6 +401,16 @@ namespace OpenViBEPlugins
boolean CGrazVisualization::process()
{
// Remove possibly dangling idle func, this construct makes sure only one idle func is registered at a time.
{
boost::mutex::scoped_lock lock(m_oIdleFuncMutex);
if(m_uiIdleFuncTag)
{
g_source_remove(m_uiIdleFuncTag);
m_uiIdleFuncTag = 0;
}
}
const IBoxIO* l_pBoxIO=getBoxAlgorithmContext()->getDynamicBoxContext();
for(uint32 chunk=0; chunk<l_pBoxIO->getInputChunkCount(0); chunk++)
......@@ -514,6 +472,12 @@ namespace OpenViBEPlugins
}
// After any possible rendering, we flush the accumulated stimuli. The default idle func is low priority, so it should be run after rendering by gtk.
{
boost::mutex::scoped_lock lock(m_oIdleFuncMutex);
m_uiIdleFuncTag = g_idle_add(flush_callback, this);
}
return true;
}
......@@ -523,7 +487,6 @@ namespace OpenViBEPlugins
{
case EGrazVisualizationState_Reference:
drawReferenceCross();
m_pStimulusSender->sendStimuli(m_ui64LastStimulation);
break;
case EGrazVisualizationState_Cue:
......@@ -626,7 +589,6 @@ namespace OpenViBEPlugins
default:
break;
}
m_pStimulusSender->sendStimuli(m_ui64LastStimulation);
}
void CGrazVisualization::drawBar()
......@@ -854,26 +816,21 @@ namespace OpenViBEPlugins
m_pRightBar = gdk_pixbuf_scale_simple(m_pOriginalBar, ui32Width, ui32Height/6, GDK_INTERP_BILINEAR);
m_pLeftBar = gdk_pixbuf_flip(m_pRightBar, true);
}
};
};
void CGrazVisualization::flushQueue(void)
{
boost::mutex::scoped_lock lock(m_oIdleFuncMutex);
/*
int main(int argc, char** argv)
{
try {
StimulusSender client("localhost", "15361");
client.sendStimuli(666);
for(size_t i=0;i<m_vStimuliQueue.size();i++)
{
m_pStimulusSender->sendStimulation(m_vStimuliQueue[i]);
}
m_vStimuliQueue.clear();
}
catch (std::exception& e)
{
std::cerr << e.what() << std::endl;
return 1;
}
// This function will be automatically removed after completion, so set to 0
m_uiIdleFuncTag = 0;
}
return 0;
}
*/
};
};
......@@ -13,12 +13,16 @@
#include <map>
#include <deque>
class StimulusSender; // fwd declare
#include <boost/thread.hpp>
#include <boost/thread/condition.hpp>
#include <boost/version.hpp>
namespace OpenViBEPlugins
{
namespace SimpleVisualisation
{
class StimulusSender; // fwd declare
enum EArrowDirection
{
EArrowDirection_None = 0,
......@@ -55,6 +59,10 @@ namespace OpenViBEPlugins
virtual void redraw();
virtual void resize(OpenViBE::uint32 ui32Width, OpenViBE::uint32 ui32Height);
public:
void flushQueue(void); // Sends all accumulated stimuli to the TCP Tagging
protected:
virtual void setStimulation(const OpenViBE::uint32 ui32StimulationIndex, const OpenViBE::uint64 ui64StimulationIdentifier, const OpenViBE::uint64 ui64StimulationDate);
......@@ -126,6 +134,10 @@ namespace OpenViBEPlugins
OpenViBE::CMatrix m_oConfusion;
// For queuing stimulations to the TCP Tagging
std::vector< OpenViBE::uint64 > m_vStimuliQueue;
guint m_uiIdleFuncTag;
boost::mutex m_oIdleFuncMutex;
StimulusSender* m_pStimulusSender;
OpenViBE::uint64 m_ui64LastStimulation;
......
......@@ -25,7 +25,7 @@ boolean StimulusSender::connect(const char* sAddress, const char* sStimulusPort)
tcp::resolver resolver(m_ioService);
// Stimulus port
std::cout << "Connecting to stimulus port [" << sAddress << " : " << sStimulusPort << "]\n";
std::cout << "Connecting to Acquisition Server's TCP Tagging [" << sAddress << " , port " << sStimulusPort << "]\n";
try
{
boost::system::error_code error;
......@@ -35,13 +35,13 @@ boolean StimulusSender::connect(const char* sAddress, const char* sStimulusPort)
m_oStimulusSocket.connect(*endpoint_iterator, error);
if(error)
{
std::cout << "Connection error: " << error << "\n";
std::cout << "-- Boost ASIO connection error: " << error << "\n";
return false;
}
}
catch (boost::system::system_error l_oError)
{
std::cout << "Issue '" << l_oError.code().message().c_str() << "' with opening connection to server\n";
std::cout << "-- Issue '" << l_oError.code().message().c_str() << "' with opening connection to server\n";
return false;
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment