Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ShopiTerm

ShopiTerm is an ecommerce platform that is unlike those you are used to. It is designed to be used from your terminal, with efficiency in mind. With a few keyboard inputs, you have bought your favorite item. The application is accessed by SSH with no need to install or run any other software than an SSH client.

This documentation gives an overview over the whole software system, how to install, configure and run it. Additionally, it provides details on how to interact with ShopiTerm and what our development workflow looks like.

Use the navigation on the side to view the respective pages.

Architecture

The ShopiTerm software system consist of multiple microservices. This page lists them and gives some detail on their implementation. For running and configuring them, see the installation and configuration pages.

External Dependencies

We use two main external dependencies in this software system: NATS and PostgreSQL.

NATS

NATS is a messaging system used by ShopiTerm for event-driven interactions. It offers a simple-to-use interface to subscribe to and publish messages on so-called subjects. This allows us to notify other services when something needs to be started or has finished.

PostgreSQL

PostgreSQL is the database management system used by ShopiTerm. Each service that needs persistence in the form of a database uses it. We use the diesel library for interfacing with PostgreSQL. This gives us strong compile-time guarantees about the correctness of our queries. All services use either an asynchronous variant of diesel or a connection pool, ensuring fast handling of concurrent requests to our system. Where necessary (e.g. when updating stock information) we use transactions to ensure that operations consisting of multiple database operations result in a consistent state.

Catalogue

The catalogue service keeps track of the items available within the store. It does not track actual stock information but rather the metadata about a given item. An item has the attributes one would expect from a shop such as name, description and price. Additionally, the catalogue offers the ability to add arbitrary tags to an item.

Warehouse

The warehouse component manages the actual stock of items for ShopiTerm. There can be any number of warehouses, each with their own stock of items. Warehouses can be created and deleted and items can be added or removed from them. In order to support the ordering process, items can be reserved at a warehouse before an order is finalized. This prevents a single item instance to be sold multiple times.

Orders

The order service handles checkout and keeps the canonical record of placed orders. It accepts a CreateOrder payload, validates the shipping address and ordered items, prices the items, reserves the required stock in the warehouse service and stores the order together with its items and shipping address in PostgreSQL.

Order creation is transactional: if persisting the order fails, any reservations are rolled back. Once the order has been written successfully, the service publishes the placed order on NATS so that the payment service can create a payment flow for it. The service also exposes endpoints to fetch an order, its items and payment information, and to mark an order as shipped. It also publishes an event to NATS when the order is marked as shipped for the notifiaction service to notify the client that the command is shipped.

Notificator

The notification service sends email confirmations to customers at key stages of the order lifecycle. It listens to NATS events published by the payment and orders services, fetches the corresponding order details from the order service, and sends customer emails via SMTP. The service sends three types of notifications: order confirmation emails when a payment is created, payment confirmation emails when payment is marked as paid, and shipping confirmation emails when an order is marked as shipped.

Payment

The payment service listens for newly created orders via NATS and creates a Taler payment for them. Taler returns a payment URI that which is announces to NATS and can then be displayed to customers for payment with their Taler wallet. All currently open payments are tracked using a NATS key-value store. Once a payment is marked as paid by Taler, the corresponding ShopiTerm order is announced to be paid via NATS.

Recommendations

TODO

TUI Client

The user interface to ShopiTerm is built using ratatui. It is exported as a library to be used the the SSH server. The interface consist of multiple views, one for each task such as catalogue browsing or doing the payment. A loop-based renderer then uses these views to assemble the user interface. Keyboard input is handled based on the currently active view.

TODO: extend?

SSH Server

TODO

Common Library

The shopiterm_common is a Rust library helping us in building the whole project. It is mainly a collection of data structures that are exchanged between services. These structures are set up for serialization and deserialization, ensuring easy re-use between services.

API Gateway

TODO

API Client

The API client is not an actual service but rather a library used by the services to exchange information. It provides the ability to make authenticated requests to the gateway and fetch all the different data structures used by the different services.

Installation

[!TIP] This explains all options in detail. To just get started with a working setup, use the compose file.

We offer three way to run ShopiTerm. The easiest to get started is using our provided containers via compose. When using NixOS, there is the option to use preconfigured NixOS modules.

[!NOTE] For the instructions in this documentation we are using Podman as the container tool, but you can use the same commands for Docker.

Containers

Each of our services is containerized and available at Quay.io. The following images are available:

ServiceImage
cataloguequay.io/asefs26-shopiterm/catalogue
notificatorquay.io/asefs26-shopiterm/notificator
ordersquay.io/asefs26-shopiterm/orders
paymentquay.io/asefs26-shopiterm/payment
recommendationsquay.io/asefs26-shopiterm/recommendations
sshquay.io/asefs26-shopiterm/ssh
warehousequay.io/asefs26-shopiterm/warehouse
krakend (Gateway)quay.io/asefs26-shopiterm/krakend

Tags are available per branch and cpu architecture. Currently, we only provide images for the amd64 architecture.

As an example, to run the latest catalogue image built of the main branch for amd64 you can use:

podman run quay.io/asefs26-shopiterm/catalogue:main-amd64 --help

This will print all available options of the catalogue service.

[!CAUTION] To function properly, most services require external dependencies and also depend on each other. We therefore provide a compose file for a simple setup.

Compose

We provide a docker-compose.yaml file for container composition using Podman / Docker Compose. It starts each service and the required dependencies, including a PostgreSQL database for each service that uses it.

To start it, simply run the following in the root of the repository:

podman compose up

[!NOTE] The preconfigured port for SSH access is 2222.

NixOS Modules

We provide NixOS modules to configure ShopiTerm on a NixOS system. Each service comes with its own module.

For usage with flakes, first import the project:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.11";
    shopiterm = {
      url = "github:asefs26/team-6";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
}

Then import the modules to configure to your NixOS configuration. The name of each service module is the same as the service itself. For example, the module for the catalogue service is shopiterm.nixosModules.catalogue.

You can check the modules for configuration details. As an example, the catalogue service can be configured as follows:

{
  services.shopiTerm.catalogue =
  let
    cataloguePort = 3001;
    warehousePort = 3002;
  in
  {
    enable = true;
    host = "[::]";
    port = cataloguePort;
    # Create a local PostgreSQL DB automatically.
    createDB = true;
    warehouse = "http://localhost:${toString warehousePort}";
  };
}

From Source

Lastly, there is the option to build the project directly from source. To do so, there are two options:

Nix

[!TIP] Installation instructions for Nix can be found on its website.

This project is developed using Nix. The project can be built by running the following:

nix build -L

The resulting binaries will be placed at result/bin.

Cargo

[!TIP] A Rust toolchain can be installed using rustup.

Alternatively, the Rust package manager Cargo can be used. Run the following for a build of all services:

cargo build --release

Configuration

All ShopiTerm services can be configured through environment variables. This page lists all available configuration options per service.

Catalogue

VariableDescriptionDefault
LISTENAddress on which to listen onlocalhost:3000
DBAddress of PostgreSQL instancepostgres:///shopiterm_catalogue?host=/var/run/postgresql
WAREHOUSEAddress of the warehouse servicehttp://localhost:3002
NATSAddress of the NATS serverlocalhost:4222

Warehouse

VariableDescriptionDefault
LISTENAddress on which to listen onlocalhost:3000
DBAddress of PostgreSQL instancepostgres:///shopiterm_warehouse?host=/var/run/postgresql
CATALOGUEAddress of the catalogue servicehttp://localhost:3001

Orders

VariableDescriptionDefault
LISTENAddress on which to listen onlocalhost:3000
DBAddress of PostgreSQL instancepostgres:///shopiterm_orders?host=/var/run/postgresql
NATSAddress of NATS instancelocalhost:4222
WAREHOUSEAddress of warehouse servicehttp://localhost:3002
CATALOGUEAddress of catalogue servicehttp://localhost:3001

Notificator

VariableDescriptionDefault
NATSAddress of the NATS serverlocalhost:4222
MAILER_HOSTAddress of the mailerlocalhost
MAILER_PORTPort of the mailer1025
ORDERAddress of order servicehttp://localhost:3003

Payment

VariableDescriptionDefault
TALER_BACKENDAddress of the Taler merchant backendhttps://backend.demo.taler.net
TALER_INSTANCEInstance on the Taler merchant backend to usesandbox
TALER_TOKENBearer token to use for Taler authorizationsecret-token:sandbox
NATSAddress of the NATS servernats://localhost:4222

Recommendations

VariableDescriptionDefault
LISTENAddress to bind the HTTP listener onlocalhost:3003
DATABASE_URLPostgreSQL connection URLrequired
CATALOGUE_URLBase URL of the catalogue servicehttp://localhost:3001
NATS_URLNATS server URL for the JetStream product-events streamnats://localhost:4222
RECOMPUTE_INTERVAL_SECSInterval in seconds between background recomputation passes (0 disables)300
WEIGHT_VIEWScore weight for a "view" interaction1.0
WEIGHT_ADD_TO_CARTScore weight for an "add_to_cart" interaction4.0
WEIGHT_PURCHASEScore weight for a "purchase" interaction10.0
PERSONALIZED_WINDOW_DAYSWindow in days of recent interactions used as seed signal for personalized recommendations30

SSH Server

VariableDescriptionDefault
LISTENAddress on which to listen for SSH connections0.0.0.0:2222
GATEWAY_URLBase URL of the gateway APIhttp://localhost:8080
HOST_KEY_FILEPath to an OpenSSH private key file for the host keyoptional
HOST_KEYPEM string for the host key (alternative to HOST_KEY_FILE)optional
DBPostgreSQL connection URLpostgres:///shopiterm_ssh?host=/var/run/postgresql
JWT_SECRETJWT secret for authenticationYS1zdHJpbmctc2VjcmV0LWF0LWxlYXN0LTI1Ni1iaXRzLWxvbmc

Usage

With ShopiTerm installed and configured, it can be used via SSH. Assuming it is configured as with the default compose file, ShopiTerm listens on port 2222.

ssh localhost -p 2222

Catalogue

TODO: how to navigate the catalogue

Cart

TODO

Checkout

TODO

Payment

With the checkout completed, the payment process is being started. As we try to use as much free and open source technology as possible, we opted for GNU Taler as our payment solution. There is no need to enter any payment details. Instead, a QR code will be generated which can be scanned by a Taler wallet. Once paid through Taler, the order is automatically confirmed.

[!NOTE] By default, ShopiTerm uses the demo backed hosted by Taler with the KUDOS demo currency. This is already configured in the official Taler mobile wallets for usage.

Adding Items

TODO

Order Overview

TODO

Development

This page describes our development workflow for building ShopiTerm.

Project Structure

ShopiTerm is a Rust project organized as a workspace with multiple components:

  • shopiterm_api_client — API client (library)
  • shopiterm_catalogue — Inventory management
  • shopiterm_client — TUI client (library)
  • shopiterm_common — Commonly shared data structures (library)
  • shopiterm_notificator — Email notifications
  • shopiterm_orders — Order processing
  • shopiterm_payment — Payment handling with Taler
  • shopiterm_recommendations — Product recommendation engine
  • shopiterm_ssh — SSH terminal interface
  • shopiterm_warehouse — Stock and warehouse management

For details on the architecture of ShopiTerm, see the corresponding page.

Prerequisites

Building

The entire project is built using a Nix flake. To build the whole project, use:

nix build

The resulting binaries are placed at result.

To build specific services, use:

nix build .#catalogue
nix build .#warehouse
nix build .#orders

[!TIP] When building via Nix, unit tests are automatically run after the build.

Formatting

We use treefmt with:

  • nixfmt — Nix code formatting
  • rustfmt — Rust code formatting

Format all code:

nix fmt

Linting

Rust linting using clippy is performed when doing a Nix check:

nix flake check -L

Testing

Unit Tests

As stated above, unit tests are automatically run on a Nix build. When having a Rust toolchain installed, they can also be run via Cargo:

cargo test --workspace

Integration Tests

Integration tests are implemented as Nix testers. These tests spin up virtual machines with the ShopiTerm services and the respective databases configured. They live in nix/testers. Inside nix/testers/lib you can find utilities for creating curl requests and mock instances of API items.

nix build .#catalogueTest
nix build .#recommendationsTest
nix build .#sshTest
nix build .#warehouseTest

[!NOTE] These integration tests are not run during a nix build but have to be specifically called.

Live System

There is a special tester running the whole software system for interactive testing:

nix run .#interactiveTest

Git Commit Messages

We enforce Conventional Commits using commitlint:

nix run .#commitlint

Continuous Integration

GitHub Actions (.github/workflows/CI.yaml) runs on every push:

JobDescription
CheckValidates commit messages, runs nix flake check (format + lint)
BuildBuilds all packages with nix build -L
TestRuns Nix integration tests
ContainerBuilds and pushes container images to registry

Containers

Each service comes with a container that can be built using Nix:

nix build .#<service>Container

The result will be an OCI image that can be imported:

podman load < result

[!NOTE] Images for each branch are pushed to Quay.io by CI.

Caching

We use Cachix for caching Nix builds. It is enabled in CI, but can also be used locally:

nix run nixpkgs#cachix use shopiterm

Workflow Summary

Here is an overview of what the workflow looks like:

# changes ...
↓
# build
nix build -L
↓
# format
nix fmt
↓
# lint & check
nix flake check -L
↓
# (when related) integration tests
nix build .#<service>Test
↓
# push
git push
↓
# CI checks all above stages (+ commit message)
↓
# CI builds & pushes containers