Web Infrastructure: Difference between revisions

From The Munich Maker Lab's Wiki
Jump to navigation Jump to search
← Older editNewer edit →
Visual
No edit summary
No edit summary
(25 intermediate revisions by the same user not shown)
Line 1: Line 1:
Some documentation on MuMaLab's web infrastructure stuff.
Some documentation on MuMaLab's web infrastructure stuff.
== Current Tasks ==
Checkout: [[Working Group IT]]


== Hosts ==
== Hosts ==
Line 10: Line 13:
=== Saturn ===
=== Saturn ===


* docker containers are started via systemd or via docker compose (configs in /data/ path)
* docker containers are started via systemd or via docker compose (configs in /data/ path) => target everything with docker compose with separated /data/ and /config/ folder.
* cronjob added for cleanup of old docker images
* cronjob added for cleanup of old docker images
* Firewall: ufw => check via ''sudo ufw status verbose''
* docker deamon resource limit via systemd slice (/etc/systemd/system/docker.slice)
* [https://github.com/fail2ban/fail2ban fail2ban] to ban hosts with too many authentication failures
* Firewall: ufw => check via ''sudo ufw status verbose'' (does not include docker. This would need addtional hacks like [https://github.com/chaifeng/ufw-docker this], but it had no benefits so far with the reverse proxy we already use)
 
==== Fail2Ban ====


== Services ==
== Services ==
Line 121: Line 122:
|saturn
|saturn
|docker
|docker
|Todo
|experimental
|https://hub.docker.com/_/influxdb
|https://hub.docker.com/_/influxdb
|-
|-
Line 137: Line 138:
|saturn
|saturn
|docker
|docker
|Todo
|experimental
|
|
|-
|-
|Elastic Stack
|Grafana Loki?
<s>Elastic Stack</s>
|logging.munichmakerlab.de
|logging.munichmakerlab.de
|Kibana Dashboard
|Grafana Loki
|saturn
|saturn
|docker
|docker
|Todo
|Todo
|
|}
=== SSO ===
Single Sign on with Authentik
Groups:
!to be done
{| class="wikitable"
|+
!Group
!Access to
!Details
|-
|Member
|
|
|-
|IT
|
|
|-
|
|
|
|}
=== Influxdbv2 ===
For storing data from sensors. Mostly for fun and testing purpose.
Data come from:
{| class="wikitable"
|+
!Source
!Bucket
!User
!Tags
|-
|[[Airrohr-NG]]
|lab-environment-data
|airrohr-service-user
|
|-
|
|
|
|
|-
|
|
|
|
|
|}
|}
Line 183: Line 238:
*ical2email. Sends reminder emails for events to mailing list, using the wordpress calendar. Python script running daily on vps02.thearrow.de
*ical2email. Sends reminder emails for events to mailing list, using the wordpress calendar. Python script running daily on vps02.thearrow.de


==Access==
*
The following people currently have admin access to the infrastructure:
*[[User:Milian|Milian]]
*[[User:Phier|Phier]]
*[[User:Tiefpunkt|tiefpunkt]]
*Adrian
 
==Migration and Optimization 2024==
We're planning to consolidate services into a standard deployment model, consolidate external services, and maybe add some new ones.
 
=== Ideas ===
Consolidate:
 
*Wiki: Containerize => Challenges: php modules; updating php/wiki; ggf. make it easier to include the plugins (maybe php compose module handling)
*Mailsystem: ??
 
Externally hosted, and to be transfered into MuMaLab Infrastructure
*Tickets (https://tickets.mumalab.org/courses/)
*Calendar -> Google Calendar -> NextCloud
*[https://tooljet.yt.gl/ ToolJet] (via [https://github.com/homeofmaking/OpenUnitState/tree/master OpenUnitState])
 
Planned Services
*NextCloud
*evtl Ticket System
*Single Sign On: e.g. login to wiki either locally or via SSO. Later only SSO. Can be used e.g. for nextcloud or other services as well
 
Details unclear
*InfoBeamer
 
===== Mailsystem =====
Should support migrating of existing data and maillinglists
 
- https://docs.mailcow.email/#what-is-mailcow-dockerized => seems to work with postfix and integration with mailman 3 seems to be possible
 
==== Lightburn VM ====
2. Licence for lightburn is already available. Would be nice to give members the chance to prepare Laser stuff remote and just come to the lab to laser.
 
Challenges: No Linux support https://forum.lightburnsoftware.com/t/linux-support-to-end-after-v1-7/144605; exposing remote desktop in secure way
 
 
Remote Systems?
 
https://guacamole.apache.org/
 
==== SSO ====
 
 
IDPs
 
#Option: [https://goauthentik.io/ Authentik]
#Option: https://git.cccv.de/uffd/uffd
#Option: ...?
 
Auth:
 
[https://blog.cubieserver.de/2022/complete-guide-to-nextcloud-oidc-authentication-with-authentik/ Complete guide to Nextcloud OIDC authentication with Authentik]
 
[https://docs.goauthentik.io/integrations/services/nextcloud/ Integrate Authentik and Nextcloud]
 
Wiki Plugins for OIDC etc.: [https://www.mediawiki.org/wiki/Extension:PluggableAuth Plugable Auth]
 
===ToDos===
{| class="wikitable"
|+
!Topic
!Tasks
!Who is on it/wants to do it?
!Notes
|-
|SSO
|
*test authentik on saturn => done
*test connecting authentik and nextcloud  => done
*authentic password reset => in progress
*migrate docker compose to /data/sso => in progress
*IT group in authentic
*test connecting wiki to authentik
*IaC configuration of nextcloud e.g. https://docs.goauthentik.io/integrations/services/nextcloud/#nextcloud-1 => setup, testing needed
*Checkout backup database and storage location of data
*check best practice and hardening advices
|Phier, Milian
|
|-
|Wiki
|
* containerize wiki and migrate to saturn => done
*update wiki and check how to handle better plugins e.g. with composer
*add integration with SSO
*check best practices and performance
|Severin
|
|-
|Lightburn Remote VM
|
* Setup second lightburn licence on VM
* Expose VM with some secure remote connection
|Phier
|
|-
|Migrate Node Red
|Old version 2.0.6
Latest: 4.0.5
 
- Open firewall (ufw) for <code>1880 => done</code>
 
- Migrate data to saturn and adjust settings for new version => done
 
- Create systemd for node red for version 4.0.5 => done
 
- Test container with new version - fix broken stuff => done
 
- Remove unused flows
 
|Milian
|https://hub.docker.com/r/nodered/node-red
|-
|Migrate Mailsetup
|
* Old setup is on mars with Postfix and Mailman 2 (prevents Debian update): Check for details and related services [[Mars]]
* Discuss what to use
* Setup on saturn some mail tool with Mailman 3
* Migrate all data to saturn: how? Lists: https://docs.mailman3.org/en/latest/migration.html
* Update Authentik and pretix mail config
* Migrate existing stuff
* check best practice and hardening advices
* Check how to better handle spam
|open
|
|-
|Migrate Ticket System
|Replace external https://tickets.mumalab.org/courses/ with Pretix instance on our server with ticket.munichmakerlab.de
 
* Setup DNS => Done
* Setup Pretix => Done
** <s>files and config copied over. Execute "docker compose up --build" in /data/pretix => issues: connection to redis and database seems not to work.  => connection issue with docker network? issue with traefik?  => redis was deactivated via config, but also database connections does not work</s> => <s>TODO remove treafik from docker compose; debug docker networking e.g. by connecting to the server and double check.</s>
* Connect to Authentic => TODO
* Setup for production => TODO
|Milian/Phier
|Setup new, without migration
|-
|Migrate Token DB
|<s>Deploy [https://tooljet.yt.gl/ ToolJet]  on our server</s> (might be obsolete and using other approach)
Existing setup https://github.com/homeofmaking/OpenUnitState/tree/master
 
Migrate existing token from TBD
Adjust Lasercutter and door(?) to this DB
 
* Setup FQDN tooljet.munichmakerlab.de => Done
* Check how existing setup is working
 
...
|open
|Contact German for old DB/Automation setup
|-
|Setup Nextcloud
|
* Setup nextcloud
* Create shared folders e.g. for password safe
* Check limitation of storage or how to add external storage
* Create calender
* Replace google calender with next cloud calender
* integrate new calender on homepage, kreativquartier, ticket system etc.
|Phier, Severin
|
|-
|Security and stability
|
*Setup Firewall => Done
*Add fail2ban => in progress, settings needs to get adjusted
*Add firewall to ansible: https://github.com/munichmakerlab/infrastructure/tree/debian-security-ansible
*Check Firewall together with docker: https://www.reddit.com/r/docker/s/RyUl8Akhy2 => https://github.com/chaifeng/ufw-docker?tab=readme-ov-file Test manually with nmap before
*<s>update docker networks for better separation?</s>
*Check backups
*Check logs and metrics
|Milian
|
|-
|Logging and Monitoring
|Setup Elastic Stack for Logging and Grafana + Prometheus for Metrics
- Setup docker compose for Grafana + Prometheus: https://grafana.com/docs/grafana-cloud/send-data/metrics/metrics-prometheus/prometheus-config-examples/docker-compose-linux/
- ELK Stack: https://www.elastic.co/blog/getting-started-with-the-elastic-stack-and-docker-compose
|Milian
|
|-
|IaC
|
* Setup Ansible in Repo => Done by Severin
*Playbook for Server => Done
*Playbook for Docker => in progress
|Milian
|
|-
|Cleanup
|Check MQTT and other IT devices. Which are still up to date, which can be fixed and which are not existent anymore.
Compare [[Network]] and [[MuMaBus]]
 
check remaining stuff in the lab, if something depends on old ports: [[MuMaBus]] ; Cleanup also acl.conf
|Adrian
|ongoing
|}
 
=== DONE ===
{| class="wikitable"
!Topic
!Tasks
!Who is on it/wants to do it?
!Notes
|-
|Update Apps
| e.g. Etherpad
|Severin
|Done
|-
|MQTT
|Migrate to saturn and update to latest version.
Connect with Adrian => https://munichmakerlab.slack.com/archives/C79T8NFU7/p1731197933279969
 
* migrate /etc/system/systemd/docker-traefik.service to use config file => done
* adjust new /data/traefik/config/traefik.yml to integrate MQTT => done
* create mosquito config /data/mqtt/ with old config and new requirements => done
* migrate db /var/lib/mosquitto/mosquitto.db => done
* create /etc/system/systemd/docker-mosquitto.service => done
* test to start new mqqt service and restart traefik => done
* add new ports to ufw => done
* add new ports to ansible ufw: <nowiki>https://github.com/munichmakerlab/infrastructure/tree/debian-security-ansible=</nowiki>> done
* change FQDN to saturn and test => Done
* Test migrated Broker => Done
|Mili
|DONE
|-
|IoT Setup Lab
|Local https://www.home-assistant.io/ setup to have a plattform for additional functions like power monitoring or controlling of the devices in the lab
 
* wipe and re-install local server with proxmox => vulpix.intern.munichmakerlab.de
* setup home-assistant
|Adrian
|Initial setup done
|-
|Backup
|Check Backup of Doorlok DB
=> old local server was removed
|Severin
|Done, was setup on proxmox
|}
[[Category:Infrastructure]]
[[Category:Infrastructure]]

Revision as of 20:46, 3 April 2025

Some documentation on MuMaLab's web infrastructure stuff.

Current Tasks

Checkout: Working Group IT

Hosts

We currently have 3 VMs at Hetzner:

  • mars.munichmakerlab.de (Mars)
  • jupiter.munichmakerlab.de
  • saturn.munichmakerlab.de

Saturn

  • docker containers are started via systemd or via docker compose (configs in /data/ path) => target everything with docker compose with separated /data/ and /config/ folder.
  • cronjob added for cleanup of old docker images
  • docker deamon resource limit via systemd slice (/etc/systemd/system/docker.slice)
  • Firewall: ufw => check via sudo ufw status verbose (does not include docker. This would need addtional hacks like this, but it had no benefits so far with the reverse proxy we already use)

Services

Service Name Hostname Functionality Server native/docker status Source
Website www.munichmakerlab.de Just Website, compare Github for details saturn docker productive Github Website
Wiki wiki.munichmakerlab.de Media wiki for knowledge sharing and documentation saturn docker productive
Nodered nodered.munichmakerlab.deAdmin: https://nodered.munichmakerlab.de/admin/ Automation like spacestatus, Slack Bots etc. satrun docker productive Node-RED
Log log.munichmakerlab.de Blog Tumblr - productive
Etherpad pad.munichmakerlab.de Colaboration text tool saturn docker productive Latest version: ether/etherpad-lite
Mailinglisten lists.munichmakerlab.de Mailman 2 mars native productive
Mail @munichmakerlab.de Mailserver

Details: Mars Version postfix: 3.4.23

mars native productive
Roombooking rooms.munichmakerlab.de Original for reserving rooms during covid jupiter docker deactivated BookedSchedular
Slack Inviter slack.munichmakerlab.de Self invite capability for our slack saturn docker productive rauchg/slackin
Space Status status.munichmakerlab.de Button in the lab to mark space as open/closed on slack/homepage saturn docker productive Github Spacestatus
Eclipse Mosquitto (MQTT) mqtt.munichmakerlab.de MQTT to use for other servicesservices like status etc. Compare MuMaBus saturn docker productive Eclipse Mosquitto
Nextcloud nextcloud.munichmakerlab.de Document sharing, calendar saturn docker experimental
Traefik Reverseproxy saturn.munichmakerlab.de/dashboard/
(might be disabled) 		Reverse proxy for other services saturn docker productive
Authentik SSO sso.munichmakerlab.de SSO for other services saturn docker experimental https://github.com/goauthentik/authentik
Tickets (old) tickets.mumalab.org Ticket system for workshops and events German - productive https://github.com/pretix/pretix
Tickets tickets.munichmakerlab.de Ticket system for workshops and events saturn docker experimental https://github.com/pretix/pretix
Wiki Staging wiki-staging.munichmakerlab.de Wiki for testing (temporary) saturn docker Todo
Influx DB influxdb.munichmakerlab.de DB for particles sensor (temporary) saturn docker experimental https://hub.docker.com/_/influxdb
ToolJet tooljet.munichmakerlab.de Store member and token, who has which safet course etc. Might be replaced by authentik directly saturn docker Todo https://github.com/ToolJet/ToolJet
Grafana & Prometheus monitoring.munichmakerlab.de Grafana Dashboard saturn docker experimental
Grafana Loki?

Elastic Stack

logging.munichmakerlab.de Grafana Loki saturn docker Todo

SSO

Single Sign on with Authentik

Groups:

!to be done

Group Access to Details
Member
IT

Influxdbv2

For storing data from sensors. Mostly for fun and testing purpose.


Data come from:

Source Bucket User Tags
Airrohr-NG lab-environment-data airrohr-service-user

Website

Static website at https://munichmakerlab.de

Wiki

MediaWiki at https://wiki.munichmakerlab.de/

  • Create your own account, needs to be confirmed by an admin

Maintenance

We currently have a bit of a spam problem, easiest way to fix it currently is to clean them up in the database directly. 

update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam, no confirmed mail address", acr_deleted = 1 where acr_email_authenticated is null and acr_rejected is null and acr_registration < now() - interval 7 day;

update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam", acr_deleted = 1 where acr_rejected is null and acr_registration < now() - interval 7 day;

update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam", acr_deleted = 1 where acr_rejected is null;

Status

Space status at https://status.munichmakerlab.de

MuMaBus

Space Automation, see MuMaBus for details

  • MQTT at saturn.munichmakerlab.de

Slack

Chat, with bridge to IRC

Additional Services

  • Calendar as iCal
  • ical2email. Sends reminder emails for events to mailing list, using the wordpress calendar. Python script running daily on vps02.thearrow.de
Retrieved from "https://wiki.munichmakerlab.de/index.php?title=Web_Infrastructure&oldid=8120"