241 lines
8.2 KiB
Markdown
241 lines
8.2 KiB
Markdown
# Language learning app
|
|
|
|
Language Learning App is a set of software packages that deliver a language learning experience for second+ language learners to help boost fluency through exposure to realistic-looking text.
|
|
|
|
## Thesis statements
|
|
|
|
1. Presentation of novel, realistic looking text and accompanying audio provide an engaging, motivating chance for language learners to increase their fluency of another language.
|
|
|
|
2. Interacting with a language in non short-form text (i.e. more than one sentence at a time) is more complex, and more beneficial, that interacting with single sentences.
|
|
|
|
3. Language learning should focus on the most "useful" words first. Traditional grouping of words (e.g. items of clothing, hobbies) used in traditional education are too abstract. Providing some level of personalisation of topic, and selection of words is important to engagement and motivation.
|
|
|
|
## Description of product
|
|
|
|
This is an app designed to help people learn a second(+) language. Initially from English. The app will start with French, Spanish, Italian, and German as the target languages. With English as the only source language.
|
|
|
|
Although spaced repetition is an effective mechanism to better remember words, showing words in context remains an important "before" step. This app adds value by providing the user with realistic-looking written and audio content in the language(s) they are learning at an appropriate level. From there, the user can identify vocabulary that they are unfamiliar with, and would like to commit to memory.
|
|
|
|
Additionally, Language Learning App treats the text-audio pair as important. Language learners don't just want to be able to read and write a language, they need to know how words sound
|
|
|
|
At present, the app doesn't have a solution to recognising speech, another important part of language learning.
|
|
|
|
## Technical Specifics
|
|
|
|
The application has a back-end written in python (fastapi), because of the Python ecosystem around data and machine learning.
|
|
|
|
The application has a web-based front end written in Svelte Kit. It will adopt progressive web app standards, to allow offline use. Due to technical complexity, and limitations, there are no plans for native app development.
|
|
|
|
The app relies on containerisation and docker to orchestrate moving parts.
|
|
|
|
Content generation relies heavily on asynchronous jobs.
|
|
|
|
The app should rely on self-hostable infrastructure as much as possible. Vendor-specific queueing and messaging protocols (e.g. AWS's SNS) are a liability.
|
|
|
|
Communication between the two is through HTTP, authenticated with JWT tokens.
|
|
|
|
## Running Locally with Docker Compose
|
|
|
|
This project supports two local runtime modes:
|
|
|
|
- Development mode: live-reload API and local MinIO storage.
|
|
- Production mode (run locally): production compose stack and Bunny-backed storage/CDN config.
|
|
|
|
### Make targets you will use most
|
|
|
|
- `make build-dev`: build images for dev stack (`docker-compose-dev.yml`).
|
|
- `make up-dev`: start dev stack in detached mode.
|
|
- `make logs-dev`: stream logs for the dev stack.
|
|
- `make up-prod`: start production stack in detached mode (`docker-compose-prod.yml`).
|
|
- `make logs-prod`: stream logs for the production stack.
|
|
- `make run-prod-locally`: build and run production stack locally with `docker-compose-local-override.yml`.
|
|
- `make down`: stop containers.
|
|
- `make migrate`: rebuild API image and run `alembic upgrade head` in the running API container.
|
|
- `make migrate-no-build`: run pending migrations without rebuilding.
|
|
|
|
### 1) Development mode (MinIO)
|
|
|
|
Create `.env` in repo root with at least:
|
|
|
|
```dotenv
|
|
POSTGRES_PASSWORD=replace_me
|
|
JWT_SECRET=replace_me
|
|
ANTHROPIC_API_KEY=replace_me
|
|
DEEPL_API_KEY=replace_me
|
|
DEEPGRAM_API_KEY=replace_me
|
|
GEMINI_API_KEY=replace_me
|
|
STORAGE_SECRET_KEY=replace_me
|
|
|
|
# Optional (have defaults in compose)
|
|
# POSTGRES_USER=langlearn
|
|
# POSTGRES_DB=langlearn
|
|
# STORAGE_ACCESS_KEY=langlearn
|
|
# STORAGE_BUCKET=langlearn
|
|
# API_PORT=8000
|
|
# FRONTEND_PORT=3000
|
|
# API_BASE_URL=http://localhost:8000
|
|
# ORIGIN=http://localhost:3000
|
|
# PUBLIC_API_BASE_URL=http://api:8000
|
|
```
|
|
|
|
Run:
|
|
|
|
```sh
|
|
make build-dev
|
|
make up-dev
|
|
make migrate-no-build
|
|
make logs-dev
|
|
```
|
|
|
|
Services in dev mode:
|
|
|
|
- API: `http://localhost:8000`
|
|
- Frontend: `http://localhost:3000`
|
|
- MinIO API: `http://localhost:9000`
|
|
- MinIO Console: `http://localhost:9001`
|
|
|
|
### 2) Production mode, locally
|
|
|
|
Create `.env.prod` in repo root. This mode uses Bunny for storage/CDN and production-like API startup.
|
|
|
|
```dotenv
|
|
POSTGRES_PASSWORD=replace_me
|
|
JWT_SECRET=replace_me
|
|
ADMIN_USER_EMAILS=admin@example.com
|
|
API_BASE_URL=http://localhost:8000
|
|
PUBLIC_API_BASE_URL=http://localhost:8000
|
|
ORIGIN=http://localhost:3000
|
|
|
|
ANTHROPIC_API_KEY=replace_me
|
|
DEEPL_API_KEY=replace_me
|
|
DEEPGRAM_API_KEY=replace_me
|
|
GEMINI_API_KEY=replace_me
|
|
|
|
TRANSACTIONAL_EMAIL_PROVIDER=stub
|
|
|
|
BUNNY_ZONE=replace_me
|
|
BUNNY_API_KEY=replace_me
|
|
BUNNY_CDN_BASE_URL=https://your-zone.b-cdn.net
|
|
BUNNY_TOKEN_AUTH_KEY=replace_me
|
|
BUNNY_STORAGE_ENDPOINT=https://storage.bunnycdn.com
|
|
|
|
# Optional (have defaults in compose)
|
|
# POSTGRES_USER=langlearn
|
|
# POSTGRES_DB=langlearn
|
|
# API_PORT=8000
|
|
# FRONTEND_PORT=3000
|
|
```
|
|
|
|
Run:
|
|
|
|
```sh
|
|
make run-prod-locally
|
|
```
|
|
|
|
This command uses:
|
|
|
|
- `docker-compose-prod.yml`
|
|
- `docker-compose-local-override.yml` (binds Postgres to `127.0.0.1` for local tooling)
|
|
|
|
## Database Access
|
|
|
|
To connect a local SQL viewer to the Postgres instance without exposing the port, use an SSH tunnel.
|
|
|
|
**Local machine** (running `docker-compose-prod.yml` locally):
|
|
|
|
Use the local override, which binds Postgres to `127.0.0.1` only:
|
|
|
|
```sh
|
|
docker compose -f docker-compose-prod.yml -f docker-compose-local-override.yml up
|
|
```
|
|
|
|
Then point your SQL viewer at `localhost:5432` directly.
|
|
|
|
**Remote machine** (`lla.thomaswilson.xyz`):
|
|
|
|
```sh
|
|
ssh -L 5432:localhost:5432 wilson@lla.thomaswilson.xyz -N
|
|
```
|
|
|
|
Then point your SQL viewer at `localhost:5432`. Run in a separate terminal and kill it when done.
|
|
|
|
## Deploying to a Self-Hosted Server
|
|
|
|
This stack is designed for VPS/self-host deployments driven by Docker Compose.
|
|
|
|
### High-level deployment flow
|
|
|
|
1. Install Docker Engine and Docker Compose plugin on the server.
|
|
2. Clone this repository on the server.
|
|
3. Create `.env.prod` in the repo root.
|
|
4. Start services with:
|
|
|
|
```sh
|
|
docker compose -f docker-compose-prod.yml --env-file .env.prod up -d --build
|
|
```
|
|
|
|
5. Check status and logs:
|
|
|
|
```sh
|
|
docker compose -f docker-compose-prod.yml --env-file .env.prod ps
|
|
docker compose -f docker-compose-prod.yml --env-file .env.prod logs -f
|
|
```
|
|
|
|
### Production environment variables
|
|
|
|
Configure these in `.env.prod`.
|
|
|
|
Core application and auth:
|
|
|
|
- `JWT_SECRET`: JWT signing secret used by API and frontend private auth logic.
|
|
- `ADMIN_USER_EMAILS`: comma-separated admin emails.
|
|
- `API_BASE_URL`: public base URL of API (used in generated links).
|
|
- `ORIGIN`: frontend origin URL.
|
|
- `PUBLIC_API_BASE_URL`: API URL exposed to frontend runtime/build.
|
|
|
|
Database:
|
|
|
|
- `POSTGRES_USER`: database username (default `langlearn` if omitted).
|
|
- `POSTGRES_PASSWORD`: database password.
|
|
- `POSTGRES_DB`: database name (default `langlearn` if omitted).
|
|
|
|
AI/translation providers:
|
|
|
|
- `ANTHROPIC_API_KEY`
|
|
- `DEEPL_API_KEY`
|
|
- `DEEPGRAM_API_KEY`
|
|
- `GEMINI_API_KEY`
|
|
|
|
Object storage and CDN (Bunny in production):
|
|
|
|
- `BUNNY_ZONE`: Bunny storage zone name.
|
|
- `BUNNY_API_KEY`: Bunny storage API key.
|
|
- `BUNNY_CDN_BASE_URL`: CDN base URL (for example `https://your-zone.b-cdn.net`).
|
|
- `BUNNY_TOKEN_AUTH_KEY`: token auth key used to sign expiring URLs.
|
|
- `BUNNY_STORAGE_ENDPOINT`: Bunny storage endpoint base URL (for example `https://storage.bunnycdn.com`).
|
|
|
|
Email delivery:
|
|
|
|
- `TRANSACTIONAL_EMAIL_PROVIDER`: `stub` or `scaleway`.
|
|
- If using Scaleway TEM, also set:
|
|
- `SCALEWAY_TEM_SECRET_KEY`
|
|
- `SCALEWAY_TEM_PROJECT_ID`
|
|
- `SCALEWAY_TEM_FROM_ADDRESS`
|
|
- `SCALEWAY_TEM_REGION` (defaults to `fr-par`)
|
|
|
|
Optional port overrides:
|
|
|
|
- `API_PORT`: host port for API container (default `8000`).
|
|
- `FRONTEND_PORT`: host port for frontend container (default `3000`).
|
|
|
|
## Technical Design
|
|
|
|
This application must remain self-hostable. It should not rely on proprietary infrastructure (e.g. AWS Lambda functions) to run. It should use Docker Compose and Makefiles to build projects and deploy them onto a local server or a VPS.
|
|
|
|
The main components so far are:
|
|
|
|
- Backend server (fastapi)
|
|
- Front end (SvelteKit)
|
|
- Object storage strategy:
|
|
- MinIO for local development
|
|
- Bunny Storage + Bunny CDN for deployed environments
|