Skip to content

Commit

Permalink
Update to Pravega 0.12 (#8)
Browse files Browse the repository at this point in the history
- Updated Readme file to work with latest Pravega
- Fixed leap mock server
- Upgrade packages
    -- Keycloak 0.12
    -- Pravega 0.12
    -- Milo Client 0.6.8
  • Loading branch information
rahulcadiga authored Nov 30, 2022
1 parent e2cbd68 commit 57f835d
Show file tree
Hide file tree
Showing 5 changed files with 46 additions and 15 deletions.
39 changes: 33 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ Pravega Sensor Collector supports the following devices:
- Linux network interface card (NIC) statistics (byte counters, packet counters, error counters, etc.)
- Phase IV Leap Wireless Gateway
- Generic CSV file import
- OPC UA Client

## About this Guide

Expand Down Expand Up @@ -194,7 +195,7 @@ Keycloak is used to authenticate to Pravega on Streaming Data Platform (SDP).
```shell
edge@edge1:~$
NAMESPACE=edge
kubectl get secret ${NAMESPACE}-pravega -n ${NAMESPACE} -o jsonpath="{.data.keycloak\.json}" | base64 -d ; echo
kubectl get secret ${NAMESPACE}-ext-pravega -n ${NAMESPACE} -o jsonpath="{.data.keycloak\.json}" | base64 -d ; echo
```

2. On the target, copy the Keycloak credentials to the file /opt/pravega-sensor-collector/conf/keycloak.json.
Expand All @@ -212,7 +213,12 @@ If the TLS Certificate Authority (CA) used by Pravega is not trusted by a well-k
edge@edge1:~/desdp$
scp ~/desdp/certs/* admin@gw:
```
OR
```
kubectl get secret pravega-controller-tls -n nautilus-pravega -o jsonpath="{.data.tls\.crt}" | base64 --decode > ~/pravega.crt
kubectl get secret keycloak-tls -n nautilus-system -o jsonpath="{.data.tls\.crt}" | base64 --decode > ~/keycloak.crt
kubectl get secret pravega-tls -n nautilus-pravega -o jsonpath="{.data.tls\.crt}" | base64 --decode > ~/pravegaAll.crt
```
2. On the target system, add the CA certificate to the operating system.
```shell
admin@gw1:~$
Expand Down Expand Up @@ -305,6 +311,9 @@ These can be defined in the parameter `LOG_FILE_INGEST_EVENT_TEMPLATE` which acc
### Write Sample Events
Edit the configuration file /opt/pravega-sensor-collector/conf/env-local.sh
(See [env-sample-telit.sh](pravega-sensor-collector/src/main/dist/conf/env-sample-telit.sh) for a sample configuration)
If using the CSV file driver, you can simulate the functionality of it by using the procedure in this section.
1. On the target device, create the file named /opt/dw/staging/Accelerometer.0000000001.tmp.
Expand Down Expand Up @@ -336,8 +345,8 @@ If using the CSV file driver, you can simulate the functionality of it by using
## Phase IV Leap Wireless Gateway Integration
See [env-sample-leap.sh](pravega-sensor-collector/src/main/dist/conf/env-sample-leap.sh) for a sample configuration
file that reads from a Leap Wireless Gateway.
Edit the configuration file /opt/pravega-sensor-collector/conf/env-local.sh
(See [env-sample-leap.sh](pravega-sensor-collector/src/main/dist/conf/env-sample-leap.sh) for a sample configuration)
The following are example records that are written to the Pravega stream.
Expand All @@ -349,6 +358,24 @@ The following are example records that are written to the Pravega stream.
{"deviceId":"000072FFFEF0000","readingTimestamp":"2021-11-24T18:40:23Z","receivedTimestamp":"2021-11-24T18:42:33.649728Z","values":[{"componentIndex":0,"sensorIndex":0,"valueIndex":0,"sensorValueDefinitionId":7,"value":23.88,"status":"Success","label":"Temperature ","iconUrl":"/images/Thermometer_16x16.png","units":""},{"componentIndex":0,"sensorIndex":0,"valueIndex":1,"sensorValueDefinitionId":8,"value":23.87,"status":"Success","label":"Temperature Weighted Average ","iconUrl":"/images/Thermometer_16x16.png","units":""},{"componentIndex":0,"sensorIndex":1,"valueIndex":0,"sensorValueDefinitionId":9,"value":1,"status":"Success","label":"Door Status (Open / Closed)","iconUrl":"/images/Switch_16x16.png","units":""},{"componentIndex":0,"sensorIndex":1,"valueIndex":1,"sensorValueDefinitionId":10,"value":0,"status":"Success","label":"Door Open Time (sec)","iconUrl":"/images/Clock_16x16.png","units":""}]}
```
## Linux network interface card (NIC) statistics
Edit the configuration file /opt/pravega-sensor-collector/conf/env-local.sh
(See [env-sample-network.sh](pravega-sensor-collector/src/main/dist/conf/env-sample-network.sh) for a sample configuration)
To get `PRAVEGA_SENSOR_COLLECTOR_NET1_NETWORK_INTERFACE` value
```shell
admin@gw1:~$
ls /sys/class/net/
```
## OPC UA Client
Edit the configuration file /opt/pravega-sensor-collector/conf/env-local.sh
(See [env-sample-opcua.sh](pravega-sensor-collector/src/main/dist/conf/env-sample-opcua.sh) for a sample configuration)
To Simulate use OPC UA Servers like Kepware or opensource servers [opc-ua-demo-server](https://github.com/digitalpetri/opc-ua-demo-server)
## Troubleshooting
### Logging
Expand Down Expand Up @@ -382,7 +409,7 @@ admin@gw1:~$
cd
git clone https://github.com/pravega/pravega
cd pravega
git checkout r0.9
git checkout r0.12
./gradlew startStandalone \
-Dcontroller.transaction.lease.count.max=2592000000 \
-Dcontroller.transaction.execution.timeBound.days=30
Expand Down Expand Up @@ -419,7 +446,7 @@ Committers of the project should use the following procedure to release a new ve
2. Commit the changes to the master branch using the normal Github pull request procedure.
3. `git tag v0.2.12`
3. `git tag v0.2.14`
4. `git push --tag`
Expand Down
6 changes: 3 additions & 3 deletions gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ junitVersion=4.12
jakartaBindVersion=2.3.2
jaxbVersion=2.3.2
javaxServletApiVersion=3.0.1
miloVersion=0.6.3
pravegaCredentialsVersion=0.9.0
pravegaVersion=0.9.0
miloVersion=0.6.8
pravegaCredentialsVersion=0.12.0
pravegaVersion=0.12.0
qosLogbackVersion=1.2.3
slf4jApiVersion=1.7.25
sqliteVersion=3.32.3
Expand Down
12 changes: 8 additions & 4 deletions pravega-sensor-collector/src/main/dist/conf/env-sample-opcua.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,18 @@
#
# http://www.apache.org/licenses/LICENSE-2.0
#
export pravega_client_auth_method=Bearer
export pravega_client_auth_loadDynamic=true
export KEYCLOAK_SERVICE_ACCOUNT_FILE=/opt/pravega-sensor-collector/conf/keycloak.json
export PRAVEGA_SENSOR_COLLECTOR_OPC1_PRAVEGA_CONTROLLER_URI=tls://pravega-controller.sdp.cluster1.sdp-demo.org:443
export PRAVEGA_SENSOR_COLLECTOR_OPC1_CLASS=io.pravega.sensor.collector.opcua.OpcUaClientDriver
export PRAVEGA_SENSOR_COLLECTOR_OPC1_SCOPE=examples
export PRAVEGA_SENSOR_COLLECTOR_OPC1_SCOPE=edge
export PRAVEGA_SENSOR_COLLECTOR_OPC1_STREAM=opc_sensor_stream
export PRAVEGA_SENSOR_COLLECTOR_OPC1_EXACTLY_ONCE=false
export PRAVEGA_SENSOR_COLLECTOR_OPC1_TRANSACTION_TIMEOUT_MINUTES=2.0
export PRAVEGA_SENSOR_COLLECTOR_OPC1_ROUTING_KEY=$(hostname)
export PRAVEGA_SENSOR_COLLECTOR_OPC1_POLL_PERIODICITY_MS=1000
export PRAVEGA_SENSOR_COLLECTOR_OPC1_ENDPOINT=opc.tcp://127.0.0.1:49320
export PRAVEGA_SENSOR_COLLECTOR_OPC1_ENDPOINT=opc.tcp://127.0.0.1:62541/milo
export PRAVEGA_SENSOR_COLLECTOR_OPC1_NAMESPACE_INDEX=2
export PRAVEGA_SENSOR_COLLECTOR_OPC1_NODE_IDENTIFIER=TestSim.Device1.Random
export PRAVEGA_SENSOR_COLLECTOR_OPC1_NODE_FILTER_REGEX='^(?!_).*'
export PRAVEGA_SENSOR_COLLECTOR_OPC1_NODE_IDENTIFIER=Dynamic/RandomInt32
export PRAVEGA_SENSOR_COLLECTOR_OPC1_NODE_FILTER_REGEX='.*'
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ public class LeapAPIMock {
* @return Grizzly HTTP server.
*/
private static HttpServer startServer() {
final ResourceConfig rc = new ResourceConfig().packages("io.pravega.sensor.collector.leap");
final ResourceConfig rc = new ResourceConfig(LeapMockResources.class);
return GrizzlyHttpServerFactory.createHttpServer(URI.create(SERVER_URI + ":" + PORT_NUM), rc);
}

Expand Down
2 changes: 1 addition & 1 deletion scripts/env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ export ENV_LOCAL_SCRIPT=$(dirname $0)/env-local.sh
if [[ -f ${ENV_LOCAL_SCRIPT} ]]; then
source ${ENV_LOCAL_SCRIPT}
fi
export APP_VERSION=${APP_VERSION:-0.2.12}
export APP_VERSION=${APP_VERSION:-0.2.14}
export GRADLE_OPTIONS="${GRADLE_OPTIONS:-"-Pversion=${APP_VERSION}"}"

0 comments on commit 57f835d

Please sign in to comment.