Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
OLMIS-6858 Reporting stack to data-pump strategy
OLMIS-6858 Reporting stack to data-pump strategy
Feel free to notify anyone else who should look at this review.

Feel free to notify anyone else who should look at this review.

There are two main connectors for each service (referencedata and requisition), one source (e.g. source-requisition) and one sink (e.g. sink-requisition). The additional sink connectors are for tab...

There are two main connectors for each service (referencedata and requisition), one source (e.g. source-requisition) and one sink (e.g. sink-requisition). The additional sink connectors are for tables that do not have a single id field defined as the primary key.

This is just to match the schema changes in the db.

This is just to match the schema changes in the db.

Script for starting up the stack locally. Not sure if it works in Linux and older versions of macOS.

Script for starting up the stack locally. Not sure if it works in Linux and older versions of macOS.

New settings to define access to the source db and the sink db (for the connectors).

New settings to define access to the source db and the sink db (for the connectors).

This is just for local dev, removing scalyr and logging to syslog, so the logs can be viewed through docker logs.

This is just for local dev, removing scalyr and logging to syslog, so the logs can be viewed through docker logs.

I don't expect we would start up with different versions for Kafka and Zookeeper, so there is just one Confluent version.

I don't expect we would start up with different versions for Kafka and Zookeeper, so there is just one Confluent version.

This is basically the schema in reference data and requisition, minus the foreign keys. Additionally, there is an additional table to define timeliness for each country, and then the materialized v...

This is basically the schema in reference data and requisition, minus the foreign keys. Additionally, there is an additional table to define timeliness for each country, and then the materialized views at the end.

These are here so that the folders exist when mounting the volume to the container.

These are here so that the folders exist when mounting the volume to the container.

Add superset patchup version to env file

So that it is set.

OLMIS-6858 Enable streaming architecture for reporting stack

This is to start the data pump feature for the reporting stack. The components of the streaming architecture are added (Kafka, Connect, etc.) and the first set of connectors (for reference data) are registered. There is one source connector for all reference data tables and one sink connector for most reference data tables. The additional sink connectors are for tables that are not identified by an id field, but rather by a combination of ids.

  1. … 6 more files in changeset.
OLMIS-6768: Updated nifi init memory allocation

OLMIS-6478: Moved hard-coded reporting stack settings to file

  1. … 15 more files in changeset.
OLMIS-6478: Moved hard-coded reporting stack settings to file

  1. … 15 more files in changeset.
Set JVM Memory initial and maximum heap size

  1. … 1 more file in changeset.
update nifi from 1.7.0 to 1.9.2

  1. … 7 more files in changeset.
Initial commit

  1. … 4 more files in changeset.
Define variables in env file and import it during preloading

  1. … 3 more files in changeset.
Define variables in env file and import it during preloading

  1. … 3 more files in changeset.
changes to .env file

changes to .env file

Set database variables

Set database variables

OLMIS-6183, simplify proxy options

Focus is on the nginx config and changing the proxy pass options. Primarly forwarding the Host header that will include the port as well as the requested address (e.g. nifi.local) and the port. This allows Nginx to be bound to ports other than 80 and 443. Further reduced the expected use-cases be assuming that Nginx as a reverse proxy will fully encapsulate Nifi and Superset, and that if a user wants to use SSL, either the termination will be done at Nginx and the configuration will include certs, or the termination may happen a level above Nginx (e.g. an AWS ELB) in which case running a server on port 443 is unneeded.

  1. … 2 more files in changeset.
OLMIS-5492, upgrade to debezium 0.9.4

OLMIS-6050 Cap the NiFi Container Resource Usage (#68)

* OLMIS-6050 Cap the NiFi container resource usage

Make the memory limit and reservation for the NiFi container

configurable

Signed-off-by: Morris Mukiri <morrismukiri@gmail.com>

* OLMIS-6050 make nifi cpu usage limit configurable

Signed-off-by: Morris Mukiri <morrismukiri@gmail.com>

* OLMIS-6050 Set NiFi JVM Heap Max

Make the JVM Heap Max environment variable configurable.

Co-authored-by: Jason Rogena <jasonrogena@gmail.com>

Signed-off-by: Morris Mukiri <morrismukiri@gmail.com>

* Update .env

  1. … 1 more file in changeset.
Update .env

Merge branch 'master' of github.com:OpenLMIS/openlmis-ref-distro into OLMIS-6050-cap-nifi-resource-usage

Signed-off-by: Morris Mukiri <morrismukiri@gmail.com>

  1. … 1 more file in changeset.
OLMIS-6050 Set NiFi JVM Heap Max

Make the JVM Heap Max environment variable configurable.

Co-authored-by: Jason Rogena <jasonrogena@gmail.com>

Signed-off-by: Morris Mukiri <morrismukiri@gmail.com>

  1. … 1 more file in changeset.