57 Semaphore Playbooks und Templates

57.1 Einbindung von Playbooks aus Repositories

57.1.1 Git als Standardquelle

Semaphore nutzt Git-Repositories als primäre Quelle für Ansible-Playbooks und ermöglicht dadurch professionelle Versionskontrolle und Kollaboration. Die Integration erfolgt über Repository-Definitionen innerhalb von Projekten.

57.1.1.1 Repository-Konfiguration

Unterstützte Git-Provider:

Authentifizierungsmethoden:

SSH-basiert:
├── Deploy-Keys (Repository-spezifisch)
├── SSH-Agent-Forwarding
└── Private-Key-Authentifizierung

HTTPS-basiert:
├── Personal Access Tokens
├── OAuth-Token
└── Username/Password (nicht empfohlen)

57.1.1.2 Repository-Setup

Grundkonfiguration:

Repository Name: ansible-infrastructure
Git URL: git@github.com:company/ansible-infrastructure.git
Branch: main
SSH Key: deploy-key-infrastructure
Access: Private Repository

Repository-Struktur für Semaphore:

ansible-infrastructure/
├── playbooks/
│   ├── webserver-deployment.yml
│   ├── database-backup.yml
│   ├── system-maintenance.yml
│   └── security-hardening.yml
├── roles/
│   ├── nginx/
│   ├── mysql/
│   ├── monitoring/
│   └── backup/
├── inventories/
│   ├── production/
│   ├── staging/
│   └── development/
├── group_vars/
├── host_vars/
├── requirements.yml              # Ansible Galaxy Dependencies
└── ansible.cfg                   # Ansible-Konfiguration

57.1.2 Unterstützung mehrerer Branches oder Tags

57.1.2.1 Branch-basierte Workflows

Environment-spezifische Branches:

Git-Repository Branches:
├── main (production-ready code)
├── staging (pre-production testing)
├── development (feature development)
└── hotfix/* (emergency fixes)

Semaphore Template-Konfiguration:
├── Production Template → main branch
├── Staging Template → staging branch
├── Development Template → development branch
└── Hotfix Template → hotfix/* pattern

57.1.2.2 Tag-basierte Releases

Release-Management mit Git-Tags:

# Git-Repository Tagging
git tag -a v1.0.0 -m "Production Release 1.0.0"
git tag -a v1.1.0-rc1 -m "Release Candidate 1.1.0"
git push origin --tags

Semaphore Template-Konfiguration:

Template: Production Deployment
├── Repository: ansible-infrastructure
├── Branch/Tag: v1.0.0
├── Playbook: playbooks/webserver-deployment.yml
└── Description: "Stable production release"

Template: Release Candidate Testing
├── Repository: ansible-infrastructure
├── Branch/Tag: v1.1.0-rc1
├── Playbook: playbooks/webserver-deployment.yml
└── Description: "Testing new features"

57.1.2.3 Dynamic Branch Selection

Template-Parameter für Branch-Wahl:

# Template Extra Variables
target_branch: "{{ branch | default('main') }}"
deployment_environment: "{{ environment }}"

# Beispiel-Ausführung
semaphore job start --template-id 1 --extra-vars="branch=hotfix/critical-fix"

57.1.3 Aktualisierung und Synchronisation

57.1.3.1 Automatische Synchronisation

Webhook-Integration:

{
  "webhook_url": "https://semaphore.company.com/api/project/1/repository/1/sync",
  "events": ["push", "tag"],
  "branches": ["main", "staging", "development"],
  "secret": "webhook-secret-token"
}

Repository-Sync-Verhalten:

  1. Git-Pull: Aktueller Branch wird synchronisiert
  2. Dependency-Installation: requirements.yml wird verarbeitet
  3. Validation: Syntax-Prüfung der Playbooks
  4. Template-Update: Verknüpfte Templates werden aktualisiert
  5. Notification: Benutzer werden über Änderungen informiert

57.1.3.2 Manuelle Synchronisation

Sync-Trigger über Web-Interface:

Project → Repositories → [Repository auswählen] → Sync Now
├── Force Sync: Überschreibt lokale Änderungen
├── Branch Selection: Wechsel zu anderem Branch
└── Status Display: Sync-Fortschritt und Fehlermeldungen

57.1.3.3 Scheduled Synchronisation

Periodische Updates:

Sync Schedule: "0 */6 * * *"  # Alle 6 Stunden
├── Branch: main
├── Auto-update Templates: Enabled
├── Notification on Error: Enabled
└── Backup before Sync: Enabled

57.1.4 Vorteile der Versionskontrolle

57.1.4.1 Nachvollziehbarkeit

Änderungshistorie:

Template Execution History:
├── Job #1234: Commit a1b2c3d (v1.0.0) - Success
├── Job #1235: Commit d4e5f6g (v1.0.1) - Failed
└── Job #1236: Commit a1b2c3d (v1.0.0) - Success (Rollback)

57.1.4.2 Kollaboration

Multi-Developer Workflows:

57.1.4.3 Rollback-Möglichkeiten

Schnelle Wiederherstellung:

Rollback-Szenario:
1. Fehlerhaftes Deployment mit v1.1.0
2. Template auf v1.0.0 zurücksetzen
3. Rollback-Playbook ausführen
4. System-Validation durchführen

57.2 Templates: Parameterisieren von Playbook-Läufen

57.2.1 Unterschied zwischen Playbooks und Templates

57.2.1.1 Ansible Playbooks

Playbooks sind statische YAML-Definitionen für Automatisierungsaufgaben:

# webserver-deployment.yml
---
- name: Deploy Web Application
  hosts: webservers
  become: yes
  vars:
    app_version: "1.0.0"
    app_port: 8080
  tasks:
    - name: Install application
      unarchive:
        src: "https://releases.company.com/app-{{ app_version }}.tar.gz"
        dest: /opt/application
        remote_src: yes

57.2.1.2 Semaphore Templates

Templates sind konfigurierte Ausführungsumgebungen für Playbooks mit:

57.2.2 Definieren von Variablen und Parametern

57.2.2.1 Template-Parameter

Template-Konfiguration:

Template Name: "Deploy Web Application"
Playbook: playbooks/webserver-deployment.yml
Inventory: production-webservers
Credentials: web-deployment-key

Extra Variables:
├── app_version: "1.2.0"
├── app_environment: "production" 
├── enable_ssl: true
├── max_connections: 1000
└── deployment_user: "deploy"

57.2.2.2 Parameterisierte Variablen

Dynamische Eingaben:

# Template Extra Variables mit Platzhaltern
app_version: "{{ version | default('latest') }}"
deployment_environment: "{{ env }}"
feature_flags: "{{ features | default('') }}"
rollback_version: "{{ previous_version | default('') }}"

# Verwendung bei Job-Ausführung
version: "1.3.0"
env: "staging"
features: "new-ui,api-v2"

57.2.2.3 Variable-Validation

Input-Validierung in Playbooks:

- name: Validate deployment parameters
  assert:
    that:
      - app_version is defined
      - app_version | regex_search('^\\d+\\.\\d+\\.\\d+$')
      - deployment_environment in ['development', 'staging', 'production']
    fail_msg: "Invalid deployment parameters provided"

57.2.3 Wiederverwendbarkeit von Templates

57.2.3.1 Template-Varianten

Basis-Template mit Variationen:

Base Template: "Application Deployment"
├── Production Variant
│   ├── Inventory: production-servers
│   ├── Variables: production-specific
│   └── Credentials: prod-deploy-key
├── Staging Variant  
│   ├── Inventory: staging-servers
│   ├── Variables: staging-specific
│   └── Credentials: staging-deploy-key
└── Development Variant
    ├── Inventory: dev-servers
    ├── Variables: dev-specific
    └── Credentials: dev-deploy-key

57.2.3.2 Template-Vererbung

Gemeinsame Basis-Konfiguration:

# Base Template Variables
common_variables:
  app_name: "web-application"
  log_level: "INFO"
  health_check_url: "/health"
  
# Environment-specific Overrides
production_variables:
  <<: *common_variables
  log_level: "WARN"
  instance_count: 3
  
staging_variables:
  <<: *common_variables
  log_level: "DEBUG"
  instance_count: 1

57.2.4 Beispiel: Template für verschiedene Umgebungen

57.2.4.1 Multi-Environment Template

Template-Definition:

Template Name: "Multi-Environment Deployment"
Repository: ansible-deployments
Playbook: playbooks/application-deploy.yml

Parameters:
├── target_environment (required)
│   ├── Values: [development, staging, production]
│   └── Default: development
├── application_version (required)
│   ├── Pattern: semver (x.y.z)
│   └── Default: latest
├── enable_monitoring (optional)
│   ├── Type: boolean
│   └── Default: true
└── deployment_strategy (optional)
    ├── Values: [rolling, blue-green, canary]
    └── Default: rolling

57.2.4.2 Environment-spezifische Konfiguration

Conditional Inventory Selection:

# Template Extra Variables
inventory_mapping:
  development: "dev-servers"
  staging: "staging-servers"  
  production: "prod-servers"

selected_inventory: "{{ inventory_mapping[target_environment] }}"

Environment-spezifische Variablen:

# Environment Configuration Matrix
environment_config:
  development:
    instance_count: 1
    resource_limits:
      memory: "512Mi"
      cpu: "0.5"
    ssl_enabled: false
    
  staging:
    instance_count: 2
    resource_limits:
      memory: "1Gi"
      cpu: "1.0"
    ssl_enabled: true
    
  production:
    instance_count: 5
    resource_limits:
      memory: "2Gi"
      cpu: "2.0"
    ssl_enabled: true
    backup_enabled: true

57.2.4.3 Template-Execution-Flow

Deployment-Pipeline:

- name: Environment-specific deployment
  include_vars: "vars/{{ target_environment }}.yml"
  
- name: Validate environment configuration
  assert:
    that:
      - target_environment in ['development', 'staging', 'production']
      - application_version is match('^\d+\.\d+\.\d+$')
      
- name: Deploy application
  include_tasks: "tasks/deploy-{{ deployment_strategy }}.yml"
  vars:
    env_config: "{{ environment_config[target_environment] }}"

57.3 Tags, Limit, Diff-Mode, Dry-Run

57.3.1 Verwendung von Tags für selektive Ausführung

57.3.1.1 Tag-Definition in Playbooks

Task-Tags:

- name: Install application packages
  package:
    name: "{{ app_packages }}"
    state: present
  tags:
    - packages
    - installation
    - security-updates

- name: Configure application
  template:
    src: app.conf.j2
    dest: /etc/app/app.conf
  tags:
    - configuration
    - config

- name: Start application service
  service:
    name: "{{ app_service }}"
    state: started
    enabled: yes
  tags:
    - service
    - startup

57.3.1.2 Template-Tag-Konfiguration

Semaphore Template Settings:

Template: "Application Management"
├── Playbook: playbooks/app-management.yml
├── Tags: "configuration,service"
├── Skip Tags: "packages"
└── Description: "Update config and restart service only"

57.3.1.3 Tag-Execution-Beispiele

Selektive Ausführung:

# Nur Konfiguration aktualisieren
--tags "configuration"

# Installation und Service-Start
--tags "installation,service"

# Alles außer Packages
--skip-tags "packages"

# Mehrere Tag-Gruppen
--tags "configuration" --tags "service"

57.3.2 Einsatz von Limit zur Host-Begrenzung

57.3.2.1 Host-Pattern-Syntax

Limit-Optionen:

# Einzelner Host
--limit "web01.company.com"

# Mehrere Hosts
--limit "web01.company.com,web02.company.com"

# Gruppe
--limit "webservers"

# Gruppenkombination
--limit "webservers:&production"

# Ausschluss
--limit "webservers:!web03.company.com"

# Pattern-Matching
--limit "web*.company.com"

# Slice-Notation
--limit "webservers[0:2]"  # Erste 3 Hosts der Gruppe

57.3.2.2 Template-Limit-Konfiguration

Semaphore Template-Einstellungen:

Template: "Rolling Update"
├── Inventory: production-all
├── Limit: "webservers[0:1]"
├── Description: "Deploy to first webserver only"
└── Use Case: "Blue-Green Deployment - Phase 1"

Template: "Emergency Patch"
├── Inventory: production-all  
├── Limit: "{{ target_hosts | default('all') }}"
├── Extra Variables: target_hosts
└── Description: "Apply patch to specified hosts"

57.3.2.3 Praktische Limit-Anwendungen

Gestaffeltes Deployment:

# Rolling Deployment Strategy
- name: Phase 1 - Deploy to subset
  hosts: "webservers[0:{{ (groups['webservers'] | length / 3) | int }}]"
  serial: 1
  
- name: Phase 2 - Deploy to remaining
  hosts: "webservers[{{ (groups['webservers'] | length / 3) | int + 1 }}:]"
  serial: 2

57.3.3 Diff-Mode zur Vorschau von Änderungen

57.3.3.1 Diff-Mode-Aktivierung

Template-Konfiguration:

Template: "Configuration Review"
├── Diff Mode: Enabled
├── Playbook: playbooks/system-config.yml
├── Purpose: "Show configuration changes before apply"
└── Recommendation: "Review diff output before production"

57.3.3.2 Diff-Output-Beispiele

File-Änderungen:

--- /etc/nginx/nginx.conf   2024-08-15 10:30:00.000000000 +0200
+++ /tmp/nginx.conf 2024-08-15 14:45:00.000000000 +0200
@@ -15,7 +15,7 @@
     server_tokens off;
     
     # Worker processes
-    worker_processes auto;
+    worker_processes 4;
     
     # Worker connections
     worker_connections 1024;

Service-Status-Änderungen:

TASK [Restart nginx service] ****************************************************
--- before
+++ after
@@ -1,4 +1,4 @@
 {
-    "state": "stopped",
+    "state": "started",
     "enabled": true
 }

57.3.3.3 Diff-Integration in Workflows

Review-Process:

- name: Generate configuration diff
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
    backup: yes
  diff: yes
  check_mode: yes
  register: config_diff

- name: Display changes for review
  debug:
    var: config_diff.diff
  when: config_diff.changed

57.3.4 Dry-Run (Check-Mode) für Tests

57.3.4.1 Check-Mode-Verhalten

Check-Mode-Charakteristika: - Keine tatsächlichen Systemänderungen - Simulation der Playbook-Ausführung - Identifikation potentieller Änderungen - Validierung von Playbook-Logik

57.3.4.2 Template-Check-Mode-Konfiguration

Semaphore Template-Settings:

Template: "Pre-Production Validation"
├── Check Mode: Enabled
├── Diff Mode: Enabled
├── Playbook: playbooks/production-deploy.yml
├── Purpose: "Validate changes before production deployment"
└── Scheduling: "Run before every production deployment"

57.3.4.3 Check-Mode-Limitationen

Module-Abhängigkeiten:

# Nicht alle Module unterstützen Check-Mode vollständig
- name: Check if service exists (works in check mode)
  service_facts:
  
- name: Restart service (limited check mode support)
  service:
    name: nginx
    state: restarted
  # May show 'changed' in check mode even without changes

57.3.4.4 Check-Mode-Best-Practices

Validierungs-Pipeline:

- name: Validation Phase
  block:
    - name: Check syntax
      include_tasks: validate-syntax.yml
      
    - name: Test connectivity  
      ping:
      
    - name: Validate configuration files
      template:
        src: "{{ item }}.j2"
        dest: "/tmp/{{ item }}"
      loop:
        - nginx.conf
        - app.conf
      check_mode: yes
      diff: yes

57.4 Umgang mit Artefakten und Logs

57.4.1 Speicherung von Ausführungsergebnissen

57.4.1.1 Job-Artefakt-Typen

Automatisch gespeicherte Artefakte:

Job Execution #1234:
├── stdout.log              # Standard-Ausgabe
├── stderr.log              # Fehler-Ausgabe  
├── ansible-facts.json      # Gesammelte Host-Facts
├── job-summary.json        # Execution-Summary
├── inventory-snapshot.yml  # Verwendetes Inventory
└── extra-vars.json         # Template-Variablen

57.4.1.2 Artefakt-Retention-Policies

Aufbewahrungsrichtlinien:

Retention Configuration:
├── Log Retention: 90 Tage
├── Artefakt Retention: 30 Tage
├── Failed Jobs: 180 Tage (extended)
├── Critical Templates: 365 Tage
└── Cleanup Schedule: Täglich um 02:00 Uhr

57.4.1.3 Custom Artefakt-Generierung

Playbook-Integration:

- name: Generate deployment report
  template:
    src: deployment-report.j2
    dest: "/tmp/deployment-{{ ansible_date_time.epoch }}.html"
  delegate_to: localhost
  
- name: Store configuration backup
  fetch:
    src: "/etc/{{ item }}"
    dest: "./backups/{{ inventory_hostname }}/{{ item }}"
    flat: yes
  loop:
    - nginx.conf
    - ssl/certificates.crt

57.4.2 Einsicht in Logs für Debugging

57.4.2.1 Log-Hierarchie

Multi-Level Logging:

Job #1234 - Deploy Web Application
├── Job-Level Logs
│   ├── Pre-execution validation
│   ├── Inventory loading
│   ├── Credential preparation
│   └── Post-execution cleanup
├── Playbook-Level Logs  
│   ├── Play execution order
│   ├── Host selection
│   └── Variable resolution
└── Task-Level Logs
    ├── Individual task execution
    ├── Module parameters
    ├── Return values
    └── Error details

57.4.2.2 Log-Verbosity-Level

Semaphore Logging-Konfiguration:

Template: "Detailed Application Deployment"
├── Verbose Level: -vvv
├── Log stdout: Enabled
├── Log stderr: Enabled  
├── Log ansible facts: Enabled
└── Debug logging: Enabled

Ansible Verbosity-Levels:

-v    : Basic output
-vv   : More detailed output
-vvv  : Debug-level output with execution details
-vvvv : Maximum verbosity with connection debugging

57.4.2.3 Real-time Log-Monitoring

Live-Log-Stream:

Job Execution Interface:
├── Real-time stdout stream
├── Color-coded output
├── Expandable task details
├── Error highlighting
├── Progress indicators
└── Execution timeline

57.4.2.4 Log-Parsing und -Filterung

Log-Analysis-Features:

Log Viewer Capabilities:
├── Search functionality
├── Filter by log level
├── Filter by host
├── Filter by task
├── Download raw logs
└── Export filtered results

57.4.3 Umgang mit sensiblen Informationen

57.4.3.1 Log-Sanitization

Automatische Bereinigung:

# Ansible no_log directive
- name: Set database password
  set_fact:
    db_password: "{{ vault_db_password }}"
  no_log: true

- name: Configure database connection
  template:
    src: database.conf.j2
    dest: /etc/app/database.conf
  no_log: true
  when: db_password is defined

57.4.3.2 Credential-Masking

Semaphore Credential-Protection:

Log Output Masking:
├── Password fields: ********
├── SSH private keys: [PROTECTED]
├── API tokens: [MASKED]
├── Certificate data: [REDACTED]
└── Custom sensitive vars: [HIDDEN]

57.4.3.3 Secure-Logging-Configuration

Template-Security-Settings:

Template: "Secure Database Setup"
├── Hide sensitive data: Enabled
├── Mask credentials: Enabled
├── Log level: Standard (no debug)
├── Retain logs: 30 days
└── Access restriction: Admins only

57.4.4 Export von Artefakten

57.4.4.1 Export-Funktionen

Verfügbare Export-Optionen:

Export Capabilities:
├── Individual job logs (TXT, JSON)
├── Multiple job comparison (CSV)
├── Execution reports (HTML, PDF)
├── Compliance reports (XML, JSON)
├── Custom artefakt bundles (ZIP)
└── API-based exports (REST)

57.4.4.2 Automated Export-Workflows

Integration in CI/CD:

# Post-deployment reporting
- name: Generate deployment summary
  uri:
    url: "{{ semaphore_api_url }}/project/{{ project_id }}/tasks/{{ task_id }}/output"
    headers:
      Authorization: "Bearer {{ api_token }}"
    method: GET
  register: deployment_logs
  delegate_to: localhost

- name: Store in artifact repository
  copy:
    content: "{{ deployment_logs.json | to_nice_json }}"
    dest: "./artifacts/deployment-{{ ansible_date_time.date }}.json"
  delegate_to: localhost

57.4.4.3 Compliance-Export

Audit-Trail-Generierung:

Compliance Report Contents:
├── Execution metadata
│   ├── User authentication
│   ├── Timestamp information
│   ├── Template configuration
│   └── Inventory details
├── Security validation
│   ├── Credential usage
│   ├── Permission verification
│   ├── Network access logs
│   └── Change documentation
└── Result documentation
    ├── Success/failure status
    ├── Applied changes
    ├── System impacts
    └── Rollback capabilities

57.4.5 Best Practices für Artefakt-Management

57.4.5.1 Strukturierte Artefakt-Organisation

Hierarchische Speicherung:

/semaphore/artifacts/
├── projects/
│   ├── web-application/
│   │   ├── 2024/08/
│   │   │   ├── job-1234/
│   │   │   └── job-1235/
│   │   └── reports/
│   │       ├── weekly/
│   │       └── monthly/
│   └── infrastructure/
│       └── 2024/08/
└── system/
    ├── backups/
    └── compliance/

57.4.5.2 Automated Cleanup-Policies

Lifecycle-Management:

# Artefakt-Cleanup-Strategy
cleanup_policies:
  job_logs:
    retention_days: 90
    cleanup_schedule: "0 2 * * 0"  # Weekly
  
  failed_jobs:
    retention_days: 180
    priority: high
    
  compliance_reports:
    retention_days: 2555  # 7 years
    archive_location: "/backup/compliance"
    
  debug_artifacts:
    retention_days: 14
    size_limit: "1GB"