|
|
[[_TOC_]]
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
When compiling Kaldi to wasm, the following libraries are
|
|
|
compiled:
|
|
|
1. CLAPACK + CBLAS + BLAS
|
|
|
2. OpenFST
|
|
|
3. Kaldi
|
|
|
|
|
|
When building OpenFST, you may be prompted to install
|
|
|
other programs for Kaldi. If this is the case, follow
|
|
|
the instructions you are given.
|
|
|
|
|
|
## Build of the matrix library
|
|
|
|
|
|
In this project, we rely on a version of CLAPACK/CBLAS/BLAS
|
|
|
which has been automatically translated from FORTRAN to C
|
|
|
via the program `f2c`.
|
|
|
|
|
|
Note that after the automatic translation, some manual
|
|
|
corrections had to be made in order to successfully compile
|
|
|
Kaldi and to pass the tests on matrix operations in Kaldi.
|
|
|
|
|
|
After a successful build of the matrix library, the following
|
|
|
static libraries are available:
|
|
|
|
|
|
```
|
|
|
clapack-wasm/
|
|
|
├── CLAPACK
|
|
|
│ ├── lapack_WA.a
|
|
|
│ ├── libcblaswr.a
|
|
|
├── f2c_BLAS-3.8.0
|
|
|
│ └── blas_WA.a
|
|
|
├── lib
|
|
|
│ └── cblas_WA.a
|
|
|
└── libf2c
|
|
|
└── libf2c.a
|
|
|
```
|
|
|
|
|
|
## Build of Kaldi
|
|
|
|
|
|
The script `install.sh` only builds the `online2bin` subdirectory
|
|
|
and its dependencies. We only build this directory as the only
|
|
|
program of interest is located there.
|
|
|
|
|
|
At the end of the build, a number of static libraries as well as
|
|
|
byte-code for all the programs under `kaldi/src/online2bin` are
|
|
|
generated. However, for the program to be usable in javascript,
|
|
|
the steps described below had to be taken.
|
|
|
|
|
|
## Build to binaries usable in javascript
|
|
|
|
|
|
### Binding C++ code
|
|
|
|
|
|
When exporting C++ classes to WASM, emscripten needs the classes to
|
|
|
be bound (cf. the [emscripten documentation][em_bind]). In our case,
|
|
|
we chose to use Embind and the following binding code:
|
|
|
|
|
|
```c++
|
|
|
#ifdef __EMSCRIPTEN__
|
|
|
|
|
|
#include <emscripten/bind.h>
|
|
|
#include <emscripten/val.h>
|
|
|
#include <iterator>
|
|
|
|
|
|
using std::vector;
|
|
|
using kaldi::int16;
|
|
|
using emscripten::val;
|
|
|
using emscripten::class_;
|
|
|
using emscripten::optional_override;
|
|
|
using emscripten::register_vector;
|
|
|
|
|
|
/* Convert JS Int16Array to C++ std::vector<kaldi::int16> without copy of data
|
|
|
*/
|
|
|
vector<int16> typed_array_to_vector(const val &int16_array) {
|
|
|
unsigned int length = int16_array["length"].as<unsigned int>();
|
|
|
vector<int16> vec(length);
|
|
|
|
|
|
val memory = val::module_property("HEAP16")["buffer"];
|
|
|
val memoryView = val::global("Int16Array").new_(memory,
|
|
|
reinterpret_cast<std::uintptr_t>(vec.data()), length);
|
|
|
|
|
|
memoryView.call<void>("set", int16_array);
|
|
|
|
|
|
return vec;
|
|
|
}
|
|
|
|
|
|
// Export class and methods of interest
|
|
|
EMSCRIPTEN_BINDINGS(asr) {
|
|
|
class_<kaldi::OnlineASR>("OnlineASR")
|
|
|
.constructor<const std::vector<std::string>& >()
|
|
|
// Inject lambda before class method call to adapt I/O types
|
|
|
.function("processBuffer", optional_override(
|
|
|
[](kaldi::OnlineASR& self, const val& int16_array) {
|
|
|
vector<int16> vect_array = typed_array_to_vector(int16_array);
|
|
|
return self.ProcessSTLVector(vect_array);
|
|
|
})
|
|
|
)
|
|
|
.function("reset", &kaldi::OnlineASR::Reset)
|
|
|
;
|
|
|
// Define JS class StringList to be understood as vector<string> in C++
|
|
|
register_vector<std::string>("StringList");
|
|
|
}
|
|
|
|
|
|
#endif
|
|
|
|
|
|
```
|
|
|
|
|
|
### Creating the javascript usable files
|
|
|
|
|
|
Given the byte-code file `kaldi/src/online2bin/dirty-preload`, the
|
|
|
javascript usable files are generated via the compiler `em++` (provided
|
|
|
by emscripten).
|
|
|
|
|
|
`em++` takes the byte-code file **with a `.bc` extension** as well as
|
|
|
compiler options to produce the file given in the `-o` option. Note
|
|
|
that the extension of the file given in the `-o` option determines
|
|
|
the kind of file that is produced.
|
|
|
|
|
|
For an example of how `em++` is used in our project, please refer to
|
|
|
`prepare_kaldi_wasm.sh`.
|
|
|
|
|
|
For more information about `em++`, please refer to [the official documentation][emcc_doc].
|
|
|
|
|
|
[em_bind]: https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html#interacting-with-code-binding-cpp
|
|
|
[emcc_doc]: https://emscripten.org/docs/tools_reference/emcc.html |
|
|
\ No newline at end of file |