dev-client.md 14.2 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251

Here are aggregated notes forming the developer documentation on the Vidjil web client.
This documentation is a work-in-progress, it is far from being as polished as the user documentation.
Help can also be found in the source code and in the commit messages.


# Client

## Installation

Run a `make` into `browser/` to get the necessary files.
This will in particular get the germline files as well as the icon files.

Opening the `browser/index.html` file is then enough to get a functionning client,
able to open `.vidjil` files with the `import/export` menu.

To work with actual data, the easiest way is to copy `js/conf.js.sample` to `js/conf.js`.
This will unlock the `patients` menu and allow your local client
to access the public server at <http://app.vidjil.org/>.

## Client API and permanent URLs

The client can be opened on a data file specified from a `data` attribute,
and optionally on an analysis file specified from a `analysis` attribute,
as in the following URLs on our test server:

  - <http://app.vidjil.org/browser/?data=test.vidjil>
  - <http://app.vidjil.org/browser/?data=test.vidjil&analysis=test.analysis>
  - <http://app.vidjil.org/browser/?data=http://app.vidjil.org/browser/test.vidjil>

Both GET and POST requests are accepted.
Note that the `browser/index.html` file and the `.vidjil/.analysis` files should be hosted on the same server.
Otherwise, the server hosting the `.vidjil/.analysis` files must accept cross-domain queries.

The client can also load data from a server (see below, requires logging), as in <http://app.vidjil.org/?set=3241&config=39>

|             |               |
| ----------- | ------------- |
| `set=xx`    | sample set id |
| `config=xx` | config id     |

Older formats (patients, run…) are also supported for compatibility but deprecated.
Moreover, the state of the client can be encoded in the URL, as in <http://app.vidjil.org/?set=3241&config=39&plot=v,size,bar&clone=11,31>

|                  |                       |
| ---------------- | --------------------- |
| `plot=x,y,m`     | plot (x axis, y axis) |
| `clone=xx,xx,xx` | selected clone ids    |

For `plot` the axis names are found in `browser/js/axes.js`. `m` is optional, and defines the type of plot (either `grid` or `bar`).

We intend to encode more parameters in the URL.

## Architecture

The Vidjil client is a set of *views* linked to a same *model*.
The model keeps the views in sync on some global properties,
most notably dealing with the selection of clones, with the clone filtering,
as well with the locus selection.

  - The model (`js/model.js`) is the main object of the Vidjil client.
    It loads and saves `.vidjil` json data (either directly from data, or from a local file, or from some url).
    It provides function to access and edit information on the clones and on the global parameters
    It keeps all the views in sync.

  - Each of the views (`Graph`, `ScatterPlot`, `List`, `Segment`) is rendered inside one or several `<div>` elements,
    and kept sync with the model. All the views are optional, and several views of the same type can be added.
    See `js/main.js` for the invocation

  - The link with the patient database/server is done with the `Database` object (`js/database.js`)

  - Other objects: `Report`, `Shortcut`
    Extends functionalities but requires elements from the full `index.html`.

## Integrating the client

### HTML and CSS

  - The `index.html` contains the `<div>` for all views and the menus

  - The CSS (`css/light.css`) is generated by `less` from `css/vidjil.less`

  - The `small_example.html` is a minimal example embedding basic HTML, CSS, as well as some data.
    As the menus are not embedded in this file, functionalities should be provided by direct calls to the models and the views.

### Javascript

  - The wonderful library `require.js` is used, so there is only one file to include
    \<script data-main="js/app.js" src="js/lib/require.js"\>\</script\>

  - `js/main.js` creates the different views and binds them to the model.
    Another option is to directly define a function named `main()`, as in `small_example.html`.

### JSON .vidjil data

Clone lists can be passed to the model through several ways:

  - directly by the user (import/export)
  - from a patient database (needs a database)
  - trough the API (see below)
  - or by directly providing data through Javascript (as in `small_example.html`)

The first three solutions need some further elements from the full `index.html`.

## Notifications

### Priority

\#<span id="browser:priority"></span>
The priority determines how the notification are shown and what action the
user should do. The priorities can be between 0 and 3.

  - 0  
    The notification is not shown
  - 1  
    The notification is shown (usually on green background) and
    automatically disappears
  - 2  
    The notification is shown (usually on yellow background) and
    automatically disappears
  - 3  
    The notification is shown (usually on red background) and doesn't
    disappear until the user clicks on it.

In the `console.log`, the field `priority` takes one of those priorities.

## Plots

### How to add something to be plotted

You want to add a dimension in the scatterplot or as a color? Read the
following.

1.  Scatterplot
    
    In [scatterPlot.js](../browser/js/scatterPlot.js), the `available_axis` object defines the dimensions that
    can be displayed. It suffices to add an entry so that it will be proposed
    in the X and Y axis. This kind of way of doing should be generalized to
    the other components.
    
    The presets are defined in the `preset` object.

2.  Color
    
    Adding a color needs slightly more work than adding a dimension in the
    scatterplot.
    
    The function `updateColor` in file [clone.js](../browser/js/clone.js) must be modified to add our color method.
    The variable `this.color` must contain a color (either in HTML or RGB, or…).
    
    Then a legend must be displayed to understand what the color represents.
    For this sake, modify the `build_info_color` method in [info.js](../browser/js/info.js) file. By
    default four spans are defined (that can be used) to display the legend:
    `span0`, …, `span3`.
    
    Finally modify the [index.html](../browser/index.html) file to add the new color method in the
    select box (which is under the `color_menu` ID).

## Classes

### Clone

1.  Info box
    
    In the info box all the fields starting with a \_ are put. Also all the
    fields under the `seg` field are displayed as soon as they have a `start` and
    `stop`. Some of them can be explicitly not displayed by filling the
    `exclude_seg_info` array in `getHtmlInfo`.

## Tests

### Code Quality

Quality of code is checked using [JSHint](http://jshint.com/), by
running `make quality` from the `browser` directory.

Install with `npm install -g jshint`

### Unit

The unit tests in the client are managed by QUnit and launched using
[nightmare](http://www.nightmarejs.org/), by launching `make unit` from the `browser/test` directory.
The tests are organised in the directory
[browser/test/QUnit/testFiles](../browser/test/QUnit/testFiles). The file [data<sub>test</sub>.js](../browser/test/QUnit/testFiles/data_test.js) contains a toy
dataset that is used in the tests.

Unit tests can be launched using a real client (instead of nightmare). It
suffices to open the file [test<sub>Qunit</sub>.html](../browser/test/QUnit/test_Qunit.html). In this HTML webpage it is
possible to see the coverage. It is important that all possible functions
are covered by unit tests. Having the coverage displayed under Firefox
needs to display the webpage using a web server for security
reasons. Under Chromium/Chrome this should work fine by just opening the
webpage.

1.  Installation
    
    Nightmare is distributed withing `node` and can therefore be installed with it.
    
    ``` bash
    apt-get install nodejs-legacy npm
    npm install nightmare -g # make -C browser/test unit will automatically
    link to global nightmare installation
    ```
    
    Note that using `nightmare` for our unit testing
    requires the installation of `xvfb`.

2.  Debugging
    
    If there is a problem with the nightmare or electron (nightmare
    dependency), you may encounter a lack of output or error messages.
    To address this issue, run:
    
    ``` bash
    cd browser/test/QUnit
    DEBUG=nightmare*,electron:* node nightmare.js
    ```

### Functional

1.  Architecture
    
    The client functional testing is done in the directory
    `browser/tests/functional`, with Watir.
    The functional tests are built using two base files:
    
      - vidjil<sub>browser</sub>.rb  
        abstracts the vidjil browser (avoid using IDs or
        class names that could change in the test). The tests must rely as
        much as possible on vidjil<sub>browser</sub>. If access to some
        data/input/menus are missing they must be addded there.
      - browser<sub>test</sub>.rb  
        prepares the environment for the tests. Each test
        file will extend this class (as can be seen in test<sub>multilocus</sub>.rb)
    
    The file `segmenter_test.rb` extends the class in `browser_test.rb` to adapt
    it to the purpose of testing the analyze autonomous app.
    
    The tests are in the files whose name matches the pattern `test*.rb`. The
    tests are launched by the script in `../launch_functional_tests` which launches
    all the files matching the previous pattern. It also backs up the test
    reports as `ci_reporter` removes them before each file is run.

2.  Installation
    
    The following instructions are for Ubuntu.
    For OS X, see <https://github.com/watir/watirbook/blob/master/manuscript/installation/mac.md>.
    
    1.  Install rvm
        
        ``` bash
Mikaël Salson's avatar
Mikaël Salson committed
252
        curl -sSL https://get.rvm.io | sudo bash
253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313
        ```
        
        Afterwards you may need to launch:
        
        ``` bash
        source /etc/profile.d/rvm.sh
        ```
    
    2.  Install ruby 2.6.1
        
        ``` bash
        rvm install 2.6.1
        ```
    
    3.  Switch to ruby 2.6.1
        
        ``` bash
        rvm use 2.6.1
        ```
    
    4.  Install necessary gems
        
        ``` bash
        gem install minitest minitest-ci watir test-unit
        ```
    
    5.  Install web browsers
        
        The Firefox version used can be set with an environment variable (see
        below). By default, the tests only work with Firefox ≤ 45. But this can be
        modified with an environment variable. All Firefox releases are [available here](https://download-installer.cdn.mozilla.net/pub/firefox/releases/%20).
        
        One can instead choose to launch functional tests using chrome. You should
        install `chromium-browser` as well as `chromium-chromedriver`. On old Chrome
        versions the `chromedriver` package may not exist. In such a case you should
        [download the ChromeDriver by yourself](https://chromedriver.storage.googleapis.com/index.html?path=2.11/) (the supported Chrome versions are
        written in the `notes.txt` file of each version).

3.  Launch client tests
    
    ``` bash
    make functional
    ```
    
    1.  Environment variables
        
        By default the tests are launched on the Firefox installed on the system.
        This can be modified by providing the `FUNCTIONAL_CLIENT_BROWSER_PATH`
        environment variable (which can contain several pathes, separated with
        spaces) to the `launch_functional_tests` script. Or, if one wants to launch
        individual test scripts, to set the `WATIR_BROWSER_PATH` environment
        variable.
        
        Other environment variables that can be specified to tune the default behaviour
        
          - `WATIR_CHROME`  
            should be set to something evaluated to `True` (*e.g.* 1) if the
            tests must be launched using Chrome
          - `WATIR_MARIONETTE`  
            should be set to something evalued to `True` (*e.g. 1*)
            if the tests must be launched on a recent Firefox version (\> 45)
314 315 316 317 318 319 320 321 322 323
            
    2.  If you have set a configuration file (browser/js/conf.js), you should remove it during the tests. The easiest way to do it is to launch these commands before and after the tests
    
        ``` bash
        # before tests
        mv browser/js/conf.js     browser/js/conf.js.bak 
        # after tests
        mv browser/js/conf.js.bak browser/js/conf.js    
        ```
            
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391
4.  Headless mode
    
    On servers without a X server the client tests can be launched in headless
    mode.
    For this sake one needs to install a few more dependencies:
    
    ``` bash
    gem install headless
    ```
    
    The virtual framebuffer X server (`xvfb`) must also be installed. Depending
    on the operating system the command will be different:
    
    ``` bash
    # On Debian/Ubuntu
    apt-get install xvfb
    # On Fedora/CentOS
    yum install xvfb
    ```
    
    Then the client tests can be launched in headless mode with:
    
    ``` bash
    make headless
    ```
    
    It is possible to view the framebuffer content of `Xvfb` using `vnc`. To do so,
    launch:
    
    1.  `x11vnc -display :99 -localhost`
    2.  `vncviewer :0`

5.  Interactive mode
    
    For debugging purposes, it may be useful to launch Watir in interactive
    mode. In that case, you should launch `irb` in the `browser/tests/functional`
    directory.
    
    Then load the file `browser_test.rb` and create a `BrowserTest`:
    
    ``` ruby
    load 'browser_test.rb'
    bt = BrowserTest.new "toto"
    
    # Load the Vidjil client with the given .vidjil file
    bt.set_browser("/doc/analysis-example.vidjil")
    ```
    
    Finally you can directly interact with the `VidjilBrowser` using the `$b`
    variable.
    
    Another way of debugging interactively is by using (and installing) the
    `ripl` gem. Then you should add, in the `.rb` file to debug:
    
    ``` ruby
    require 'ripl'
    ```
    
    Then if you want to stop launch an `irb` arrived at a given point in the
    code, the following command must be inserted in the code:
    
    ``` ruby
    Ripl.start :binding => binding
    ```
    
    If you have to launch `irb` on a remote server without X (only using `Xvfb`)
    you may be interested to use the [redirection over SSH](https://en.wikipedia.org/wiki/Xvfb#Remote_control_over_SSH).