Web Infrastructure: Difference between revisions
No edit summary |
No edit summary |
||
(2 intermediate revisions by the same user not shown) | |||
Line 45: | Line 45: | ||
|- | |- | ||
| Mailinglisten||[https://lists.munichmakerlab.de lists.munichmakerlab.de] | | Mailinglisten||[https://lists.munichmakerlab.de lists.munichmakerlab.de] | ||
| | |Mailman 2||mars||native||productive | ||
| | |||
|- | |||
|Mail | |||
|@munichmakerlab.de | |||
|Mailserver | |||
Details: [[Mars]] | |||
Version postfix: 3.4.23 | |||
|mars | |||
|native | |||
|productive | |||
| | | | ||
|- | |- | ||
Line 73: | Line 83: | ||
|Reverse proxy for other services||saturn||docker||productive | |Reverse proxy for other services||saturn||docker||productive | ||
| | | | ||
|- | |||
|Authentik SSO | |||
|sso.munichmakerlab.de | |||
|SSO for other services | |||
|saturn | |||
|docker | |||
|experimental | |||
|https://github.com/goauthentik/authentik | |||
|- | |- | ||
|Tickets (old) | |Tickets (old) | ||
Line 154: | Line 172: | ||
===MuMaBus=== | ===MuMaBus=== | ||
Space Automation, see [[MuMaBus]] for details | Space Automation, see [[MuMaBus]] for details | ||
*MQTT at | *MQTT at saturn.munichmakerlab.de | ||
===Slack === | ===Slack === | ||
Line 193: | Line 211: | ||
Details unclear | Details unclear | ||
*InfoBeamer | *InfoBeamer | ||
===== Mailsystem ===== | |||
Should support migrating of existing data and maillinglists | |||
- https://docs.mailcow.email/#what-is-mailcow-dockerized => seems to work with postfix and integration with mailman 3 seems to be possible | |||
==== Lightburn VM ==== | ==== Lightburn VM ==== | ||
Line 234: | Line 257: | ||
*test connecting authentik and nextcloud => done | *test connecting authentik and nextcloud => done | ||
*authentic password reset => in progress | *authentic password reset => in progress | ||
*migrate docker compose to /data/sso => in progress | |||
*IT group in authentic | *IT group in authentic | ||
*test connecting wiki to authentik | *test connecting wiki to authentik | ||
*IaC configuration of nextcloud e.g. https://docs.goauthentik.io/integrations/services/nextcloud/#nextcloud-1 | *IaC configuration of nextcloud e.g. https://docs.goauthentik.io/integrations/services/nextcloud/#nextcloud-1 => setup, testing needed | ||
*Checkout backup database and storage location of data | *Checkout backup database and storage location of data | ||
|Phier | *check best practice and hardening advices | ||
|Phier, Milian | |||
| | | | ||
|- | |- | ||
Line 244: | Line 269: | ||
| | | | ||
* containerize wiki and migrate to saturn => done | * containerize wiki and migrate to saturn => done | ||
*update wiki | *update wiki and check how to handle better plugins e.g. with composer | ||
*add integration with SSO | |||
*check best practices and performance | |||
|Severin | |Severin | ||
| | | | ||
Line 274: | Line 301: | ||
|Migrate Mailsetup | |Migrate Mailsetup | ||
| | | | ||
* Old setup is on Mailman 2 | * Old setup is on mars with Postfix and Mailman 2 (prevents Debian update): Check for details and related services [[Mars]] | ||
* Setup on saturn with Mailman 3 | * Discuss what to use | ||
* Setup on saturn some mail tool with Mailman 3 | |||
* Migrate all data to saturn: how? Lists: https://docs.mailman3.org/en/latest/migration.html | |||
* Update Authentik and pretix mail config | * Update Authentik and pretix mail config | ||
* Migrate existing stuff | * Migrate existing stuff | ||
* check best practice and hardening advices | |||
|open | |open | ||
| | | | ||
Line 310: | Line 340: | ||
* Setup nextcloud | * Setup nextcloud | ||
* Create shared folders e.g. for password safe | * Create shared folders e.g. for password safe | ||
* Check limitation of storage or how to add external storage | |||
* Create calender | * Create calender | ||
* Replace google calender with next cloud calender | * Replace google calender with next cloud calender | ||
Line 321: | Line 352: | ||
*Add fail2ban => in progress, settings needs to get adjusted | *Add fail2ban => in progress, settings needs to get adjusted | ||
*Add firewall to ansible: https://github.com/munichmakerlab/infrastructure/tree/debian-security-ansible | *Add firewall to ansible: https://github.com/munichmakerlab/infrastructure/tree/debian-security-ansible | ||
*Check Firewall together with docker: https://www.reddit.com/r/docker/s/RyUl8Akhy2 | |||
*update docker networks for better separation? | *update docker networks for better separation? | ||
*Check backups | *Check backups | ||
*Check logs and metrics | |||
|Milian | |Milian | ||
| | | | ||
Line 328: | Line 361: | ||
|Logging and Monitoring | |Logging and Monitoring | ||
|Setup Elastic Stack for Logging and Grafana + Prometheus for Metrics | |Setup Elastic Stack for Logging and Grafana + Prometheus for Metrics | ||
- Setup docker compose for Grafana + Prometheus: https://grafana.com/docs/grafana-cloud/send-data/metrics/metrics-prometheus/prometheus-config-examples/docker-compose-linux/ | |||
- ELK Stack: https://www.elastic.co/blog/getting-started-with-the-elastic-stack-and-docker-compose | |||
|Milian | |Milian | ||
| | | |
Latest revision as of 21:42, 6 January 2025
Some documentation on MuMaLab's web infrastructure stuff.
Hosts
We currently have 3 VMs at Hetzner:
- mars.munichmakerlab.de (Mars)
- jupiter.munichmakerlab.de
- saturn.munichmakerlab.de
Saturn
- docker containers are started via systemd or via docker compose (configs in /data/ path)
- cronjob added for cleanup of old docker images
- Firewall: ufw => check via sudo ufw status verbose
- fail2ban to ban hosts with too many authentication failures
Fail2Ban
Services
Service Name | Hostname | Functionality | Server | native/docker | status | Source |
---|---|---|---|---|---|---|
Website | www.munichmakerlab.de | Just Website, compare Github for details | saturn | docker | productive | Github Website |
Wiki | wiki.munichmakerlab.de | Media wiki for knowledge sharing and documentation | saturn | docker | productive | |
Nodered | nodered.munichmakerlab.deAdmin: https://nodered.munichmakerlab.de/admin/ | Automation like spacestatus, Slack Bots etc. | satrun | docker | productive | Node-RED |
Log | log.munichmakerlab.de | Blog | Tumblr | - | productive | |
Etherpad | pad.munichmakerlab.de | Colaboration text tool | saturn | docker | productive | Latest version: ether/etherpad-lite |
Mailinglisten | lists.munichmakerlab.de | Mailman 2 | mars | native | productive | |
@munichmakerlab.de | Mailserver
Details: Mars Version postfix: 3.4.23 |
mars | native | productive | ||
Roombooking | rooms.munichmakerlab.de | Original for reserving rooms during covid | jupiter | docker | deactivated | BookedSchedular |
Slack Inviter | slack.munichmakerlab.de | Self invite capability for our slack | saturn | docker | productive | rauchg/slackin |
Space Status | status.munichmakerlab.de | Button in the lab to mark space as open/closed on slack/homepage | saturn | docker | productive | Github Spacestatus |
Eclipse Mosquitto (MQTT) | mqtt.munichmakerlab.de | MQTT to use for other servicesservices like status etc. Compare MuMaBus | saturn | docker | productive | Eclipse Mosquitto |
Nextcloud | nextcloud.munichmakerlab.de | Document sharing, calendar | saturn | docker | experimental | |
Traefik Reverseproxy | saturn.munichmakerlab.de/dashboard/ (might be disabled) |
Reverse proxy for other services | saturn | docker | productive | |
Authentik SSO | sso.munichmakerlab.de | SSO for other services | saturn | docker | experimental | https://github.com/goauthentik/authentik |
Tickets (old) | tickets.mumalab.org | Ticket system for workshops and events | German | - | productive | https://github.com/pretix/pretix |
Tickets | tickets.munichmakerlab.de | Ticket system for workshops and events | saturn | docker | experimental | https://github.com/pretix/pretix |
Wiki Staging | wiki-staging.munichmakerlab.de | Wiki for testing (temporary) | saturn | docker | Todo | |
Influx DB | influxdb.munichmakerlab.de | DB for particles sensor (temporary) | saturn | docker | Todo | https://hub.docker.com/_/influxdb |
ToolJet | tooljet.munichmakerlab.de | Store member and token, who has which safet course etc. Might be replaced by authentik directly | saturn | docker | Todo | https://github.com/ToolJet/ToolJet |
Grafana & Prometheus | Todo | |||||
Elastic Stack | Todo |
Website
Static website at https://munichmakerlab.de
Wiki
MediaWiki at https://wiki.munichmakerlab.de/
- Create your own account, needs to be confirmed by an admin
Maintenance
We currently have a bit of a spam problem, easiest way to fix it currently is to clean them up in the database directly.
update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam, no confirmed mail address", acr_deleted = 1 where acr_email_authenticated is null and acr_rejected is null and acr_registration < now() - interval 7 day; update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam", acr_deleted = 1 where acr_rejected is null and acr_registration < now() - interval 7 day; update mw_account_requests set acr_rejected = DATE_FORMAT(NOW(),"%Y%m%d%H%i%S"), acr_user = 1, acr_comment = "Spam", acr_deleted = 1 where acr_rejected is null;
Status
Space status at https://status.munichmakerlab.de
- Details at StartYourEngines
MuMaBus
Space Automation, see MuMaBus for details
- MQTT at saturn.munichmakerlab.de
Slack
Chat, with bridge to IRC
- Application in itself is SaaS. Talk to tarwin or tiefpunkt
- IRC bridge is powered by RelayBot, hosted on ???
Additional Services
- Calendar as iCal
- ical2email. Sends reminder emails for events to mailing list, using the wordpress calendar. Python script running daily on vps02.thearrow.de
Access
The following people currently have admin access to the infrastructure:
Migration and Optimization 2024
We're planning to consolidate services into a standard deployment model, consolidate external services, and maybe add some new ones.
Ideas
Consolidate:
- Wiki: Containerize => Challenges: php modules; updating php/wiki; ggf. make it easier to include the plugins (maybe php compose module handling)
- Mailsystem: ??
Externally hosted, and to be transfered into MuMaLab Infrastructure
- Tickets (https://tickets.mumalab.org/courses/)
- Calendar -> Google Calendar -> NextCloud
- ToolJet (via OpenUnitState)
Planned Services
- NextCloud
- evtl Ticket System
- Single Sign On: e.g. login to wiki either locally or via SSO. Later only SSO. Can be used e.g. for nextcloud or other services as well
Details unclear
- InfoBeamer
Mailsystem
Should support migrating of existing data and maillinglists
- https://docs.mailcow.email/#what-is-mailcow-dockerized => seems to work with postfix and integration with mailman 3 seems to be possible
Lightburn VM
2. Licence for lightburn is already available. Would be nice to give members the chance to prepare Laser stuff remote and just come to the lab to laser.
Challenges: No Linux support https://forum.lightburnsoftware.com/t/linux-support-to-end-after-v1-7/144605; exposing remote desktop in secure way
Remote Systems?
SSO
IDPs
- Option: Authentik
- Option: https://git.cccv.de/uffd/uffd
- Option: ...?
Auth:
Complete guide to Nextcloud OIDC authentication with Authentik
Integrate Authentik and Nextcloud
Wiki Plugins for OIDC etc.: Plugable Auth
ToDos
Topic | Tasks | Who is on it/wants to do it? | Notes |
---|---|---|---|
SSO |
|
Phier, Milian | |
Wiki |
|
Severin | |
Lightburn Remote VM |
|
Phier | |
Migrate Node Red | Old version 2.0.6
Latest: 4.0.5 - Open firewall (ufw) for - Migrate data to saturn and adjust settings for new version => done - Create systemd for node red for version 4.0.5 => done - Test container with new version - fix broken stuff => done - Remove unused flows |
Milian | https://hub.docker.com/r/nodered/node-red |
Migrate Mailsetup |
|
open | |
Migrate Ticket System | Replace external https://tickets.mumalab.org/courses/ with Pretix instance on our server with ticket.munichmakerlab.de
|
Milian/Phier | Setup new, without migration |
Migrate Token DB | Existing setup https://github.com/homeofmaking/OpenUnitState/tree/master Migrate existing token from TBD Adjust Lasercutter and door(?) to this DB
... |
open | Contact German for old DB/Automation setup |
Setup Nextcloud |
|
Phier, Severin | |
Security and stability |
|
Milian | |
Logging and Monitoring | Setup Elastic Stack for Logging and Grafana + Prometheus for Metrics
- Setup docker compose for Grafana + Prometheus: https://grafana.com/docs/grafana-cloud/send-data/metrics/metrics-prometheus/prometheus-config-examples/docker-compose-linux/ - ELK Stack: https://www.elastic.co/blog/getting-started-with-the-elastic-stack-and-docker-compose |
Milian | |
IaC |
|
Milian | |
Cleanup | Check MQTT and other IT devices. Which are still up to date, which can be fixed and which are not existent anymore.
check remaining stuff in the lab, if something depends on old ports: MuMaBus ; Cleanup also acl.conf |
Adrian | ongoing |
DONE
Topic | Tasks | Who is on it/wants to do it? | Notes |
---|---|---|---|
Update Apps | e.g. Etherpad | Severin | Done |
MQTT | Migrate to saturn and update to latest version.
Connect with Adrian => https://munichmakerlab.slack.com/archives/C79T8NFU7/p1731197933279969
|
Mili | DONE |
IoT Setup Lab | Local https://www.home-assistant.io/ setup to have a plattform for additional functions like power monitoring or controlling of the devices in the lab
|
Adrian | Initial setup done |
Backup | Check Backup of Doorlok DB
=> old local server was removed |
Severin | Done, was setup on proxmox |