1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
|
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>API overview — Beam SDK 1.1.0 documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="API reference" href="api_reference.html" />
<link rel="prev" title="Getting started" href="getting_started.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" style="background: #48447E" >
<a href="index.html" class="icon icon-home"> Beam SDK
</a>
<div class="version">
1.1.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting started</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">API overview</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#architecture">Architecture</a></li>
<li class="toctree-l2"><a class="reference internal" href="#reference-frames-and-units-of-tracking-data">Reference frames and units of tracking data</a></li>
<li class="toctree-l2"><a class="reference internal" href="#head-pose-reference-frame">Head pose reference frame</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tracking-data-definitions">Tracking data definitions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="api_reference.html">API reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="redistribution.html">Redistributing a Beam client application</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">Beam SDK</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home"></a> »</li>
<li>API overview</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/api_overview.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="toctree-wrapper compound">
</div>
<div class="section" id="api-overview">
<h1>API overview<a class="headerlink" href="#api-overview" title="Permalink to this headline">¶</a></h1>
<div class="section" id="architecture">
<h2>Architecture<a class="headerlink" href="#architecture" title="Permalink to this headline">¶</a></h2>
<p>The API of Beam SDK exposes head and eye tracking data in real time.</p>
<p>Beam SDK works by establishing a <em>connection</em> between the Beam eye tracking application by Eyeware, and your own code which uses the API described in these pages.
This is a <strong>server-client model</strong>.
Beam acts as the server, and you write the client which consumes the tracking data published by the server, allowing you to build your own eye tracking-enabled application.
There can be multiple clients connected to the same Beam server at one time.</p>
<p>The access to the tracking data is provided by the <code class="docutils literal notranslate"><span class="pre">TrackerClient</span></code> class.
This is the main entry point of the API, available in all supported languages.
For language-specific information about <code class="docutils literal notranslate"><span class="pre">TrackerClient</span></code>, check out the <a class="reference internal" href="api_reference.html#api-reference"><span class="std std-ref">API reference</span></a>.</p>
<p>In general, a <code class="docutils literal notranslate"><span class="pre">TrackerClient</span></code> object creates a connection between the Beam application and your code.
In the current version of Beam SDK, this connection is made via a TCP port over a network.
Constructing a <code class="docutils literal notranslate"><span class="pre">TrackerClient</span></code> object without arguments sets a default hostname and port which work fine in many configurations.
However, it is possible to set a specific hostname and port, depending on your setup and network.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The Beam application and your client program communicate data over TCP sockets.
Most commonly, they are designed to be running on the same PC.
In scenarios in which they are running on different PCs, they need to be on the same network (e.g., share wifi), and the IP of the PC running Beam needs to be specified in the client program (<code class="docutils literal notranslate"><span class="pre">hostname</span></code> argument).</p>
</div>
</div>
<div class="section" id="reference-frames-and-units-of-tracking-data">
<h2>Reference frames and units of tracking data<a class="headerlink" href="#reference-frames-and-units-of-tracking-data" title="Permalink to this headline">¶</a></h2>
<p>Beam SDK gives tracking information about:</p>
<ul class="simple">
<li><p>the head of the person who is being tracked (<strong>head tracking</strong>), in 3D;</p></li>
<li><p>the pixel of the computer screen that the person is gazing at (<strong>screen gaze tracking</strong>), in 2D.</p></li>
</ul>
<p>Head tracking information is expressed with respect to a <strong>World Coordinate System</strong> (WCS), or root frame, positioned on the center of your computer screen.
The head pose reference frame is positioned on the <strong>nose tip</strong>: see <a class="reference internal" href="#head-pose-reference-frame"><span class="std std-ref">Head pose reference frame</span></a> for illustrations.
Head tracking positions are expressed as physical distances, in <strong>meters</strong>.</p>
<p>Screen gaze coordinates are expressed in <strong>pixels</strong> (integer units) with respect to the top-left corner of the screen, which has coordinates <code class="docutils literal notranslate"><span class="pre"><0,</span> <span class="pre">0></span></code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The screen that is considered for screen gaze tracking is the display in which the Beam application is capable of showing the gaze bubble.
That is, the main display in the operating system.</p>
</div>
<p>The diagram below shows the WCS at the center of the screen (xyz axes represented in red-green-blue, respectively), as well as the 2D frame to represent screen pixels at the top-left corner (xy axes represented in red-green).</p>
<a class="reference internal image-reference" href="_images/beam_sdk_wcs.png"><img alt="Representation of the World Coordinate System" class="align-center" src="_images/beam_sdk_wcs.png" style="width: 600px;" /></a>
</div>
<div class="section" id="head-pose-reference-frame">
<h2>Head pose reference frame<a class="headerlink" href="#head-pose-reference-frame" title="Permalink to this headline">¶</a></h2>
<p>The head tracking information exposed by Beam SDK includes a rigid transform (rotation <code class="docutils literal notranslate"><span class="pre">R</span></code> and translation <code class="docutils literal notranslate"><span class="pre">t</span></code>) from the World Coordinate System frame to the <strong>head frame</strong>, positioned on the person’s <strong>nose tip</strong>.</p>
<p>Here is a lateral view that shows the <code class="docutils literal notranslate"><span class="pre">R,t</span></code> transform, as well as the xyz axes of the WCS and of the head frame:</p>
<a class="reference internal image-reference" href="_images/head_transform_isometric_rt_axes_wcs.png"><img alt="Lateral view of the head transform" class="align-center" src="_images/head_transform_isometric_rt_axes_wcs.png" style="width: 600px;" /></a>
<p>Frontal view showing the head axes from the point of view of the person:</p>
<a class="reference internal image-reference" href="_images/head_transform_frontal_axes.png"><img alt="Frontal view of the head frame" class="align-center" src="_images/head_transform_frontal_axes.png" style="width: 300px;" /></a>
<p>Top view showing the transform, the WCS frame at the screen center and the head frame on the nose tip:</p>
<a class="reference internal image-reference" href="_images/head_transform_top_rt_axes.png"><img alt="Top view of the head transform" class="align-center" src="_images/head_transform_top_rt_axes.png" style="width: 500px;" /></a>
</div>
<div class="section" id="tracking-data-definitions">
<h2>Tracking data definitions<a class="headerlink" href="#tracking-data-definitions" title="Permalink to this headline">¶</a></h2>
<p>Here is the meaning of the head and eye tracking fields exposed by Beam SDK, in all supported languages:</p>
<ul class="simple">
<li><p><strong>Head Pose</strong>: 3D head tracking measurements</p>
<ul>
<li><p><em>Lost track</em>: whether head tracking is lost or not for the current frame;</p></li>
<li><p><em>Session ID</em>: numeric identifier of the current, uninterrupted, tracking session; <a class="footnote-reference brackets" href="#fn-sessionid" id="id1">1</a></p></li>
<li><p><em>Rotation</em>: rotation matrix of the person’s nose tip, with respect to the World Coordinate System; <a class="footnote-reference brackets" href="#fn-rotation" id="id2">2</a></p></li>
<li><p><em>Translation</em>: translation vector of the person’s nose tip in meters, with respect to the World Coordinate System.</p></li>
</ul>
</li>
<li><p><strong>Gaze on Screen</strong>: 2D screen gaze tracking measurements</p>
<ul>
<li><p><em>Lost track</em>: whether gaze tracking is lost or not for the current frame;</p></li>
<li><p><em>Screen ID</em>: numeric identifier of the screen being looked at; <a class="footnote-reference brackets" href="#fn-screenid" id="id3">3</a></p></li>
<li><p><em>Lost track</em>: whether gaze tracking is lost or not for the current frame;</p></li>
<li><p><em>Coordinates</em>: horizontal and vertical pixel coordinates of the gazed point, respectively. Counted from the top-left pixel of the screen;</p></li>
<li><p><em>Confidence</em>: realibility level of the gaze tracking result.</p></li>
</ul>
</li>
</ul>
<dl class="footnote brackets">
<dt class="label" id="fn-sessionid"><span class="brackets"><a class="fn-backref" href="#id1">1</a></span></dt>
<dd><p>If a person is lost or not tracked for at least three seconds, the <em>Session ID</em> will change. When a person starts being tracked again, he or she will automatically be assigned a new <em>Session ID</em> number.</p>
</dd>
<dt class="label" id="fn-rotation"><span class="brackets"><a class="fn-backref" href="#id2">2</a></span></dt>
<dd><p>Rotation matrices are computed by Beam SDK from a triplet of rotation angles in radians along the zxy axes of the World Coordinate System, respectively. For further information, look up <a class="reference external" href="https://en.wikipedia.org/wiki/Euler_angles">Euler angles</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/Rotation_matrix">Rotation matrix</a>.</p>
</dd>
<dt class="label" id="fn-screenid"><span class="brackets"><a class="fn-backref" href="#id3">3</a></span></dt>
<dd><p>In the current version, Beam SDK supports single-screen setups. The <em>Screen ID</em> number is always zero.</p>
</dd>
</dl>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="api_reference.html" class="btn btn-neutral float-right" title="API reference" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
<a href="getting_started.html" class="btn btn-neutral float-left" title="Getting started" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright 2021, Eyeware Tech SA.
</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
|
