Privory Omega Core

The core service is the monolithic backend of the Privory business application. The application follows a 3-tier architectural design composed of controllers, a (business) service layer and a layer for database access with JPA repositories.

3-tier architecture of the Privory Omega backend

Table of Contents

1. Prerequisites
2. Local Development
3. Building and Testing
4. File Encryption System
5. Badge.js Integration
6. Production Deployment
7. Scaleway Deployment Guide
8. Troubleshooting

---

Prerequisites

Required Software

• Java: JDK 21-Temurin or Corretto 21
• Docker: Rancher Desktop with dockerd as container runtime
  • Ensure BIOS has virtualization enabled if you have Docker issues
• IDE: IntelliJ IDEA (Community or Ultimate recommended)
• API Testing: Postman or similar tool
• Build Tool: Gradle wrapper (included, no configuration needed)

Verifying Installation

java -version   # Should show Java 21
docker --version
./gradlew --version

---

Local Development

Starting the Local Environment

# Start PostgreSQL + LocalStack (S3) via Docker Compose
./gradlew runLocal

# Or start dependencies only
./gradlew composeUp

# Access application at http://localhost:8080

Environment Variables for Local Development

# Master key for file encryption (REQUIRED)
export PRIVORY_MASTER_KEY_V1="$(openssl rand -base64 32)"

# Badge.js build-time URL (REQUIRED for badge functionality)
export OMEGA_BADGE_PATH="http://localhost:8080"

Database Access (Local)

# Connection details for local PostgreSQL
Host: localhost
Port: 5432
Database: core_database
Username: postgres
Password: postgres

# Using psql client
psql "postgresql://postgres:postgres@localhost:5432/core_database"

Local S3 (LocalStack)

# LocalStack S3 endpoint
http://localhost:4566

# Test with AWS CLI
aws s3 ls --endpoint-url=http://localhost:4566

---

Building and Testing

Build Commands

# Full build with tests
./gradlew build

# Build without tests
./gradlew build -x test

# Build JAR only (output: build/libs/privory-omega.jar)
./gradlew bootJar

# Clean build
./gradlew clean build

Running Tests

# Run all tests
./gradlew test

# Run specific test class
./gradlew test --tests "EncryptionServiceImplTest"

# Generate code coverage report
./gradlew jacocoTestReport
# Report location: build/reports/jacoco/test/html/index.html

Running the Application

# Run with local profile
./gradlew run

# Run with specific profile
java -jar build/libs/privory-omega.jar --spring.profiles.active=cloud

# With debug logging
java -jar build/libs/privory-omega.jar --spring.profiles.active=cloud --debug

---

File Encryption System

Overview

All uploaded files are automatically encrypted at rest using AES-256-GCM authenticated encryption.

Two-Tier Encryption Architecture:

1. File Encryption: Each file is encrypted with a unique 256-bit file key
2. Key Encryption: File keys are encrypted with a master key before storage

Master Key Setup

1. Generate Master Key

# Generate a 256-bit AES key encoded as Base64
openssl rand -base64 32

# Output example: 3q2+796tbu904BUqSY3h1Q7fJ8yD5N8vM9KXwAbCdE0=

Why Base64?

• AES keys are 32 bytes of random binary data
• Environment variables can only store text
• Base64 converts binary → safe ASCII text (44 characters)
• The application decodes Base64 → 32 bytes when loading the key

2. Set Environment Variable

For Local Development:

export PRIVORY_MASTER_KEY_V1="3q2+796tbu904BUqSY3h1Q7fJ8yD5N8vM9KXwAbCdE0="

For Production (Scaleway/Server):

# Add to /etc/environment
sudo nano /etc/environment

# Add line:
PRIVORY_MASTER_KEY_V1="your-base64-encoded-key-here"

3. Verify Master Key Loading

Check application startup logs:

Master key metadata already exists for version 1
Master key v1 is available and ready

Master Key Versioning

The system supports key rotation for zero-downtime updates:

# Current key (active)
export PRIVORY_MASTER_KEY_V1="old-key"

# New key (for rotation)
export PRIVORY_MASTER_KEY_V2="new-key"

# Switch active version (optional, defaults to v1)
export PRIVORY_MASTER_KEY_CURRENT_VERSION="2"

After rotation:

• New files use v2
• Old files still work with v1
• Both keys must remain available

Application Configuration

# application.properties
privory.file-encryption.enabled=true
privory.file-encryption.algorithm=AES/GCM/NoPadding
privory.file-encryption.key-size=256
privory.file-encryption.iv-size=96
privory.file-encryption.tag-size=128

Security Notes

• ✅ Master keys are NEVER stored in the database or codebase
• ✅ Each file has a unique encryption key
• ✅ GCM mode provides tamper detection
• ✅ Keys are only in memory during encryption/decryption
• ❌ NEVER commit master keys to git
• ❌ NEVER expose master keys in logs

---

Badge.js Integration

Overview

The badge.js file is an embeddable widget that customers can add to their websites. The API URL is injected at build time, not runtime.

Critical Build-Time Requirement

⚠️ IMPORTANT: OMEGA_BADGE_PATH must be set BEFORE building, not just when running the application.

Why This Is Critical

The Gradle build process (in build.gradle.kts) replaces ${apiUrl} in badge.js with the value from OMEGA_BADGE_PATH:

// During build, this reads the environment variable
val badgeHostUrl = System.getenv("OMEGA_BADGE_PATH") ?: "http://localhost:8080/"

// And replaces ${apiUrl} in badge.js
filesMatching("static/badge.js") {
    filter { line -> line.replace("\${apiUrl}", badgeHostUrl) }
}

Setting Up for Different Environments

Local Development:

export OMEGA_BADGE_PATH="http://localhost:8080"
./gradlew clean build

Staging/Production:

export OMEGA_BADGE_PATH="https://staging.privory.eu"
./gradlew clean build

# Then deploy the built JAR

CI/CD Pipeline:

# GitLab CI, GitHub Actions, etc.
env:
  OMEGA_BADGE_PATH: "https://production.privory.eu"

build:
  script:
    - ./gradlew clean build

Verifying the Badge URL

After building, check the generated badge.js:

# Extract the JAR
unzip -p build/libs/privory-omega.jar BOOT-INF/classes/static/badge.js | grep "apiUrl:"

# Should show:
#   apiUrl: 'https://staging.privory.eu',

# NOT:
#   apiUrl: '${OMEGA_BADGE_PATH:http://localhost:8080/}',

Badge Embed Code

Modern format (with caching):

<script src="https://staging.privory.eu/badge.js?v=1.0.0"
        data-badge-id="YOUR-BADGE-ID"
        async></script>

Optional placeholder for STATIC mode:

<div data-privory-badge="YOUR-BADGE-ID"></div>

Badge Caching

• badge.js: Cached for 24 hours
• Version parameter (?v=1.0.0) for cache busting
• Update privory.omega.badge.script-version when changing badge.js

---

Production Deployment

Pre-Deployment Checklist

1. Environment Variables (Build-Time)

Set these BEFORE building:

# Badge API URL (CRITICAL - build-time only)
export OMEGA_BADGE_PATH="https://your-domain.com"

2. Environment Variables (Runtime)

Set these on the production server:

# File encryption master key (REQUIRED)
export PRIVORY_MASTER_KEY_V1="your-base64-key-from-openssl-rand"

# Database credentials
export DB_BUSINESS_PASSWORD="your-business-password"
export DB_MIGRATION_PASSWORD="your-migration-password"

# S3/Object Storage credentials
export AWS_S3_ACCESS_KEY="your-s3-access-key"
export AWS_S3_SECRET_KEY="your-s3-secret-key"

# Optional: Master key version (defaults to 1)
export PRIVORY_MASTER_KEY_CURRENT_VERSION="1"

3. Build Application

# With environment variables set
./gradlew clean build

# Verify JAR was created
ls -lh build/libs/privory-omega.jar

4. Deploy JAR

# Upload to server
scp build/libs/privory-omega.jar user@server:/opt/privory/

# On server, run with production profile
java -jar /opt/privory/privory-omega.jar --spring.profiles.active=cloud

Health Checks

# Application health
curl http://your-domain:8080/actuator/health

# Expected response
{"status":"UP"}

---

Scaleway Deployment Guide

Architecture Overview

┌─────────────────────────────────────────────────────────┐
│                        Internet                          │
└────────────┬──────────────────────┬─────────────────────┘
             │                      │
             │                      │
    ┌────────▼────────┐    ┌────────▼─────────────┐
    │   S3 Bucket     │    │  Application Server  │
    │   (nl-ams)      │    │  (Public IP)         │
    │                 │    │  172.16.1.20         │
    │  Public HTTPS   │    └──────────┬───────────┘
    │  Endpoint       │               │
    │                 │               │ Private Network
    │  Secured by     │               │ 172.16.1.0/24
    │  Access Keys    │               │
    └─────────────────┘    ┌──────────▼───────────┐
                           │  PostgreSQL DB       │
                           │  (NO Public IP)      │
                           │  172.16.1.10         │
                           │                      │
                           │  Only accessible     │
                           │  from VPC            │
                           └──────────────────────┘

Step-by-Step Setup

1. Create VPC

Scaleway Console → VPC → Create VPC

Name: privory-staging-vpc
Region: nl-ams (Amsterdam)
Tags: environment:staging, project:privory

2. Create Private Network

Scaleway Console → VPC → Private Networks → Create

Name: privory-staging-network
Region: nl-ams
VPC: privory-staging-vpc
IPAM: Regional IPAM
Subnet: 172.16.0.0/16

3. Create PostgreSQL Database

Scaleway Console → Managed Databases → PostgreSQL → Create

Basic Settings:

Engine: PostgreSQL 15
Name: privory-staging-db
Node Type: DB-DEV-S (2 vCPU, 2GB RAM)

Network Configuration (CRITICAL):

☑️ Attach to Private Network: privory-staging-network
☐ Enable Public Endpoint ← DO NOT ENABLE

Database Configuration:

Database: core_database
Username: privory_admin
Password: [Generate strong password]
Backup: Automatic daily backups

Note the Private IP: e.g., 172.16.0.5

4. Initialize Database

Use Scaleway Console → SQL Editor with /database/scaleway-init.sql:

-- Replace placeholder passwords first
CREATE USER core_business WITH PASSWORD 'your-strong-password';
CREATE USER core_migration WITH PASSWORD 'your-strong-password';

GRANT ALL PRIVILEGES ON DATABASE core_database TO core_business;
GRANT ALL PRIVILEGES ON DATABASE core_database TO core_migration;

Verify users:

SELECT usename FROM pg_user WHERE usename LIKE 'core_%';

5. Create S3 Bucket

Scaleway Console → Object Storage → Create Bucket

Name: privory-omega-staging
Region: nl-ams
Visibility: Private

Create API Access Key:

1. Go to IAM → API Keys → Generate API Key
2. Name: privory-omega-staging-s3
3. Copy Access Key and Secret Key

Configure Bucket Policy:

⚠️ CRITICAL: Include BOTH the bucket AND objects (/*)

{
  "Version": "2023-04-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "SCW": "application_id:YOUR-APPLICATION-ID"
    },
    "Action": "*",
    "Resource": [
      "privory-omega-staging",
      "privory-omega-staging/*"
    ]
  }]
}

Without /*, you can read the bucket but NOT write objects!

6. Create Application Server

Scaleway Console → Instances → Create

Name: privory-staging-app
Type: DEV1-S (2 vCPU, 2GB RAM)
Image: Ubuntu 22.04 LTS
Region: nl-ams

Network:

☑️ Attach to Private Network: privory-staging-network
☑️ Enable Public IP (for external access)

Security Group:

Port 22 (SSH): YOUR_IP_ONLY
Port 8080 (HTTP): 0.0.0.0/0
Port 443 (HTTPS): 0.0.0.0/0

7. Configure Application Server

SSH into server:

ssh root@<PUBLIC_IP>

Install Java:

sudo apt update
sudo apt install openjdk-21-jdk -y
java -version  # Verify

Set Environment Variables:

sudo nano /etc/environment

# Add these lines:
PRIVORY_MASTER_KEY_V1="your-base64-master-key-here"
DB_BUSINESS_PASSWORD="your-business-password"
DB_MIGRATION_PASSWORD="your-migration-password"
AWS_S3_ACCESS_KEY="your-scaleway-s3-access-key"
AWS_S3_SECRET_KEY="your-scaleway-s3-secret-key"
OMEGA_BADGE_PATH="https://staging.privory.eu"

Create application directory:

mkdir -p /opt/privory
cd /opt/privory

8. Create Application Properties

Create application-cloud.properties on server:

# ============================================================================
# DATABASE CONFIGURATION (Private Network)
# ============================================================================
spring.datasource.url=jdbc:postgresql://172.16.0.5:5432/core_database?sslmode=require
spring.datasource.username=core_business
spring.datasource.password=${DB_BUSINESS_PASSWORD}

# Connection Pool (optimized for staging)
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.idle-timeout=600000
spring.datasource.hikari.max-lifetime=1800000

# ============================================================================
# FLYWAY MIGRATION
# ============================================================================
spring.flyway.enabled=true
spring.flyway.user=core_migration
spring.flyway.password=${DB_MIGRATION_PASSWORD}
spring.flyway.locations=classpath:db/migration
spring.flyway.baseline-on-migrate=true

# ============================================================================
# S3 CONFIGURATION (Scaleway Object Storage)
# ============================================================================
aws.s3.endpoint=https://s3.nl-ams.scw.cloud
aws.s3.req-signing-region=nl-ams
aws.s3.internal-bucket=privory-omega-staging
aws.s3.credentials.access-key=${AWS_S3_ACCESS_KEY}
aws.s3.credentials.secret-key=${AWS_S3_SECRET_KEY}

# ============================================================================
# FILE ENCRYPTION
# ============================================================================
privory.file-encryption.enabled=true
# Master key loaded from PRIVORY_MASTER_KEY_V1 environment variable

# ============================================================================
# APPLICATION SETTINGS
# ============================================================================
privory.environment.img-path=https://s3.nl-ams.scw.cloud/privory-omega-staging/
privory.environment.name=staging
privory.omega.badge.host-url=${OMEGA_BADGE_PATH}

# ============================================================================
# LOGGING
# ============================================================================
logging.level.root=INFO
logging.level.de.privory.omega=DEBUG
logging.level.software.amazon.awssdk=WARN

9. Build and Deploy

On your local machine:

# Set build-time environment variable
export OMEGA_BADGE_PATH="https://staging.privory.eu"

# Build with production settings
./gradlew clean build

# Upload to server
scp build/libs/privory-omega.jar root@<PUBLIC_IP>:/opt/privory/
scp application-cloud.properties root@<PUBLIC_IP>:/opt/privory/

On the server:

cd /opt/privory

# Test run (foreground)
java -jar privory-omega.jar --spring.profiles.active=cloud

# Check logs for:
# - "Master key v1 is available and ready"
# - "Started CoreApplication in X seconds"
# - No database connection errors
# - No S3 connection errors

10. Create Systemd Service (Optional)

Create /etc/systemd/system/privory-omega.service:

[Unit]
Description=Privory Omega Core
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/privory
ExecStart=/usr/bin/java -jar /opt/privory/privory-omega.jar --spring.profiles.active=cloud
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal

# Load environment variables
EnvironmentFile=/etc/environment

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable privory-omega
sudo systemctl start privory-omega

# Check status
sudo systemctl status privory-omega

# View logs
sudo journalctl -u privory-omega -f

Testing Production Deployment

1. Database Connectivity

# From app server
psql "postgresql://core_business:PASSWORD@172.16.0.5:5432/core_database?sslmode=require"

2. S3 Connectivity

# From app server
aws s3 ls s3://privory-omega-staging --endpoint-url=https://s3.nl-ams.scw.cloud

3. Application Health

# From anywhere
curl http://<PUBLIC_IP>:8080/actuator/health

# Expected: {"status":"UP"}

4. Badge Functionality

# Test badge API
curl http://<PUBLIC_IP>:8080/v1/badge/YOUR-BADGE-ID

# Should return badge configuration JSON

Security Checklist

• PostgreSQL has NO public IP
• PostgreSQL only accessible via Private Network (172.16.x.x)
• Application server has Public IP for users + Private IP for DB
• S3 bucket policy includes BOTH bucket and bucket/*
• S3 access keys stored as environment variables (not in code)
• Master encryption key is Base64-encoded and stored securely
• DB passwords are strong (20+ characters, mixed case, numbers, symbols)
• SSL/TLS enabled for database (sslmode=require)
• Security Group restricts SSH to your IP only
• Application runs with --spring.profiles.active=cloud

Why S3 Remains Public

S3 Object Storage cannot be attached to Private Networks at Scaleway (this is by design):

• ✅ HTTPS-encrypted connections
• ✅ Authentication via Access Keys
• ✅ Files are additionally encrypted by the application (AES-256-GCM)
• ✅ Same architecture as AWS S3
• ✅ Bucket policies control access

PostgreSQL needs VPC protection because it contains sensitive structured data and would be vulnerable to SQL injection if publicly exposed.

---

Troubleshooting

Badge Shows "Badge not found or inactive"

Check browser Network tab:

• What URL is being called?
• Is it calling http://localhost:... or the correct domain?

If calling localhost:

• Badge.js was built without OMEGA_BADGE_PATH set
• Solution:

  export OMEGA_BADGE_PATH="https://your-domain.com"
  ./gradlew clean build
  # Redeploy
  

If calling correct URL but still 404:

• Badge doesn't exist in database, OR
• Badge is marked as active=false
• Check database: SELECT * FROM badge WHERE badge_id = 'your-id';

File Upload Fails with "Access Denied (403)"

S3 bucket policy is missing objects wildcard:

Check bucket policy includes:

"Resource": [
  "bucket-name",
  "bucket-name/*"  ← This line is critical
]

File Upload Fails with "Bad Request (400)"

Scaleway S3 compatibility issue:

Ensure AwsConfig.java disables chunked encoding:

.serviceConfiguration(S3Configuration.builder()
    .checksumValidationEnabled(false)
    .chunkedEncodingEnabled(false)
    .build())

Master Key Not Loading

Check startup logs for:

Master key v1 is available and ready

If missing:

1. Verify PRIVORY_MASTER_KEY_V1 is set:

   echo $PRIVORY_MASTER_KEY_V1
   

2. Verify it's Base64 encoded (44 characters):

   echo $PRIVORY_MASTER_KEY_V1 | wc -c  # Should be 45 (44 + newline)
   

3. Regenerate if needed:

   openssl rand -base64 32
   

Database Connection Fails

From application server, test manually:

psql "postgresql://core_business:PASSWORD@172.16.0.5:5432/core_database?sslmode=require"

Common issues:

• Wrong password → Check DB_BUSINESS_PASSWORD
• Wrong IP → Check database private IP in Scaleway Console
• Not on same VPC → Verify both server and DB are in privory-staging-network
• SSL error → Ensure sslmode=require in connection string

Tests Failing After Encryption Changes

If seeing AEADBadTagException:

• Tests expect old behavior (streaming CipherInputStream)
• Update test expectations to match new behavior (immediate cipher.doFinal())
• See EncryptionServiceImplTest.java for examples

Application Starts But Can't Access Pages

Check Vaadin compilation:

./gradlew vaadinPrepareFrontend

Check for errors in logs:

# Look for Vaadin-related errors
grep -i "vaadin" build/logs/application.log

---

Additional Resources

• File Encryption Details: See documentation/FILE_ENCRYPTION.md
• Badge Caching: See BADGE_CACHING.md
• Development Guide: See CLAUDE.md
• Scaleway Docs: https://www.scaleway.com/en/docs/
• Spring Boot Docs: https://spring.io/projects/spring-boot
• Vaadin Docs: https://vaadin.com/docs

---

File Upload & Storage System

The application supports flexible file upload and storage with different configurations for Serverless and Docker environments.

Key Features

1. Hybrid Processing: In-memory for small files, disk-based for large files
2. Serverless Compatible: Pure in-memory mode for environments without persistent storage
3. Configurable Limits: Adjustable memory thresholds for different workloads
4. Secure Encryption: AES-256-GCM encryption for all stored files
5. Automatic Cleanup: Temporary files are automatically removed

Configuration Options

Serverless Environment (Scaleway)

# Force in-memory mode
privory.temp-storage.force-in-memory=true
privory.temp-storage.directory=null
privory.temp-storage.memory-threshold=20971520  # 20MB

Behavior: All files processed in memory, files >20MB rejected with HTTP 413 error.

Docker Environment

# Hybrid processing
privory.temp-storage.force-in-memory=false
privory.temp-storage.directory=/tmp/privory-uploads
privory.temp-storage.memory-threshold=52428800  # 50MB

Docker Compose:

services:
  omega:
    volumes:
      - privory-temp-storage:/tmp/privory-uploads

Behavior: Files <50MB in memory, larger files use temporary disk storage.

File Upload Limits

Environment	Max File Size	Behavior
Serverless	20MB (configurable)	Hard limit, files rejected if too large
Docker	Unlimited (disk space)	Small files in memory, large files on disk

Encryption

• Algorithm: AES-256-GCM
• Key Management: Base64-encoded keys with optional master key encryption
• Security: Authenticated encryption with random IVs

Migration

Serverless → Docker:

1. Create volume: docker volume create privory-temp-storage
2. Update config: Set forceInMemory=false and directory=/tmp/privory-uploads
3. Mount volume and restart

Docker → Serverless:

1. Update config: Set forceInMemory=true and directory=null
2. Reduce memoryThreshold to match container memory
3. Restart and monitor

Monitoring

Log Messages:

• Using in-memory mode - Serverless mode active
• Uploading file of size X bytes - Normal operation
• File upload rejected - File exceeds limit
• Created temporary directory - Disk mode initialized

Best Practices

1. Serverless: Keep memoryThreshold < 30% of container memory
2. Docker: Mount volume for temporary files
3. Monitor: Track file sizes and upload patterns
4. Test: Verify with production-like file sizes

For detailed documentation, see:

• FILE_UPLOAD_LIMITS.md
• SERVERLESS_CONFIGURATION.md
• TEMP_STORAGE_MIGRATION.md

---

Support

For issues or questions:

1. Check this README first
2. Check relevant documentation files
3. Check application logs
4. Contact the development team

Common Log Locations:

• Local: build/logs/
• Production: /var/log/syslog or journalctl -u privory-omega