Due to a scheduled upgrade to version 14.10, GitLab will be unavailabe on Monday 30.05., from 19:00 until 20:00.

README.md 944 Bytes
Newer Older
Jonas Waeber's avatar
Jonas Waeber committed
1
2
## SFTP Reader & Validator

Jonas Waeber's avatar
Jonas Waeber committed
3
4
Checks the files present in the `app.directory` folder on the sFTP server.

Jonas Waeber's avatar
Jonas Waeber committed
5
6
[Confluence Doku](https://memobase.atlassian.net/wiki/spaces/TBAS/pages/29196525/Service%2BsFTP%2BReader%2BValidator)

Jonas Waeber's avatar
Jonas Waeber committed
7
8
9
10
11
### Current Features

- Check if files are CSV or TSV
- Check if CSV and TSV files are formatted correctly.

Jonas Waeber's avatar
Jonas Waeber committed
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
### Requirements 

- Access to a Kafka Server
- Access to a SFTP Server
- Set environment variable $SFTP_PASSWORD
- Set environment variable $HOST_KEY_VERIFIER (fingerprint is posted in the logs when running the connection for the first time.)

### Configuration

```yaml
sftp:
  host: <host-name>
  port: "22"
  user: <user-name>
  password: ${SFTP_PASSWORD:?nopass}
  fingerprint: ${HOST_KEY_VERIFIER:?nofinger}
app:
  directory: ./test_institution_1/test_record_set_1
kafka:
  producer:
    bootstrap.servers: localhost:9092
    client.id: sftp-reader-p1-j1
  topic:
    out: process-1-job-1-sftp-reader
```