Mentions légales du service

Skip to content
Snippets Groups Projects
MANOLESCU Ioana's avatar
MANOLESCU Ioana authored
d36950e6
History

Version GitHub license


ESTOCADA

Handy Links

  1. Download ESTOCADA Source code
  2. Code Requirements and Build
  3. System Prerequisites
  4. Description of Used Hardware
  5. Supported Native Stores Query Languages
  6. Adding New Model/Language Pair
  7. ESTOCADA MDD Tutorial
  8. Tutorial: ESTOCADA Quick Actions
  9. Testing
  10. Documentation
  11. Code Format
  12. Paper Reference
  13. Contact

Download ESTOCADA

$ git clone https://gitlab.inria.fr/cedar/estocada-public.git

Code Requirements and Build

Requirements

  • A Unix-ish environment (Linux, Mac OS X). NOTE: the code build was tested on Mac OS X and CentOS7 Linux.
  • Apache Maven 3.6.1 or newer (Download and Install)
  • Java version 1.8

Used Libraries

Instructions for the code build

The code contains four main components:

  • estocada-commons
  • estocada-PACB
  • estocada-main
  • estoccada-engine-Tatooine

Code build (skipping tests). This will automatically build the source code, all related dependencies and generate APIs documentation. Note: documenting the code is work in progress.

$ chmod +x build-skip-tests.sh
$./build-skip-tests.sh

Code build (enabling tests). This will automatically build the source code, all related dependencies, run the tests and generate APIs documentation.

$ chmod +x build-tests.sh
$./build-tests.sh

System Prerequisites

AsterixDB v0.9.3

[common]
log.level=INFO
compiler.framesize=64KB
compiler.sortmemory=64MB
compiler.groupmemory=128MB
compiler.joinmemory=256MB
compiler.textsearchmemory=256MB
log.dir=logs/
storage.max.active.writable.datasets=2

[nc]
jvm.args=-Xmx12g
txn.log.dir=txnlog
iodevices=io0,io1,io2,io3
command=asterixnc
storage.buffercache.pagesize=128KB
storage.buffercache.size=8GB
storage.memorycomponent.globalbudget=1GB
storage.memorycomponent.pagesize=64KB
[nc/1]
address=localhost

[cc]
address=localhost

MongoDB v4.0.3

PostgreSQL v9.6

SparkSQL v2.4.6

wget https://archive.apache.org/dist/spark/spark-2.4.6/spark-2.4.6.tgz
tar zxvf spark-2.4.6.tgz
cd spark-2.4.6/
./build/mvn -DskipTests clean package

Solr v6.0

wget https://archive.apache.org/dist/lucene/solr/6.0.0/solr-6.0.0.tgz
tar zxvf solr-6.0.0.tgz
cd solr-6.0.0/
# Running Solr Server
./solr start -e cloud
# Stop Running Solr Server
./solr stop -all

BigDAWG

cd bigdawg/
mvn clean install -DskipTests

Description of Used Hardware

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 40
On-line CPU(s) list: 0-39
Thread(s) per core: 2
Core(s) per socket: 10
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
Stepping: 1
CPU MHz: 2893.212
CPU max MHz: 3400.0000
CPU min MHz: 1200.0000
BogoMIPS: 4799.55
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 25600K
NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38
NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,3
Memory: 123G RAM
OS: CentOS Linux release 7.9.2009 (Core)

Supported Native Stores Query Languages

  • For each supported native store, you can find the supported fragment of the language in .g4 file under ESTOCADA compiler component estocada.compiler.model.{X} , where X is the model/language notation (e.g., aj,pj,sj,rk etc).

Adding New Model/Language Pair

  • To add a new model/language pair, you need to do the following steps:
  1. Declare the relational schema and constraints used to capture the model relationally under src/main/resources/{NameOfTheModel}. You can check src/main/resources/json directory for an example.
  2. Provide {NewModelAnnotation}.properties file for each model configuration. For example, defining fresh_variable_name_prefix for the new model. Check existing files for an example.
  3. Create a new package {NewModelAnnotation}for the new model/language pair under estocada.compiler.model.
  4. Add the supported grammar in .g4 file (We use the ANTLR tool for generating ASTs) under estocada.compiler.model.{NewModelAnnotation} and implement the encoder of the language. Check existing encoders for an example.
  5. Implement the decoder (translate a relatioanl query into a store's native language) under estocada.rewriting.decoder.{NewModelAnnotation}.

ESTOCADA MDD Tutorial

Check out the ESTOCADA MDD Tutorial.

ESTOCADA Quick Actions

Check out the tutorial for some quick actions using ESTOCADA intermmediate abstraction!

Testing

  • We added severl Unit Tests for each mode/language pair (encoder+decoder).
  • estocada.test.framework.EncoderTestSuite under src/test/java/runs all encoder Unit Tests
  • estocada.test.framework.DecoderTestSuite under src/test/java/runs all decoder Unit Tests

Documentation

cd documentation/
  • ESTOCADA Code documentation: sites/apidocs/help-doc.html. ESTOCADA APIs documentation is automatically generated from the code using mvn javadoc:javadoc. To read the doc, open help-doc.html using your default browser. ** Detailed Documentation is Work in Progress**

Code Format

# Eclipse IDE:
Step 1: Eclipse > Preferences > Java > Code Style > Formatter > import > ESTOCADACodeFormatProfile.XML
Step 2: Eclipse > Preferences > Java > Editor:
1- Select "Perform the selected actions on save"
2- Select "Format source code" and "Format all lines"
3- Select "Organize imports"
Step 3: Select "Apply and Close"
# IntelliJ IDE:
Step 1: IntelliJ > Preferences > Editor > Code Style > Java > Scheme > Import Scheme > Eclipse XML Profile > ESTOADACodeFormatProfile.XML
Step 2: Select "Apply", then "OK".

Paper Reference

If you use ESTOCADA, please cite our SIGMOD'19 and PVLDB'20 papers

@inproceedings{DBLP:conf/sigmod/AlotaibiBDMZ19,
  author    = {Rana Alotaibi and
               Damian Bursztyn and
               Alin Deutsch and
               Ioana Manolescu and
               Stamatis Zampetakis},
  title     = {Towards Scalable Hybrid Stores: Constraint-Based Rewriting to the
               Rescue},
  booktitle = {Proceedings of the 2019 International Conference on Management of
               Data, {SIGMOD} Conference 2019, Amsterdam, The Netherlands, June 30
               - July 5, 2019},
  pages     = {1660--1677},
  publisher = {{ACM}},
  year      = {2019}
}
@article{DBLP:journals/pvldb/AlotaibiCDLMY20,
  author    = {Rana Alotaibi and
               Bogdan Cautis and
               Alin Deutsch and
               Moustafa Latrache and
               Ioana Manolescu and
               Yifei Yang},
  title     = {{ESTOCADA:} Towards Scalable Polystore Systems},
  journal   = {Proc. {VLDB} Endow.},
  volume    = {13},
  number    = {12},
  pages     = {2949--2952},
  year      = {2020}
}

Contact

Email: ralotaib@eng.ucsd.edu