Skip to main content

PunchFlow Testing Framework

🎯 Revolutionary Approach: No Plugins Required!​

PunchFlow's Universal Testing Framework represents a paradigm shift in PunchOut testing. Unlike traditional solutions that require shop-specific plugins, our framework uses native shop APIs to perform comprehensive testing without any modifications to the target systems.

🌟 Key Benefits​

Zero Shop Modifications​

  • No Plugin Installation: Works with shops out-of-the-box
  • No Code Changes: Uses existing shop APIs
  • No Maintenance Overhead: Updates handled by PunchFlow

Comprehensive Testing​

  • End-to-End Validation: Complete PunchOut flow testing
  • Multi-Protocol Support: cXML, OCI, and custom protocols
  • Real Buyer Simulation: Mock systems for SAP Ariba, Coupa, etc.

Real-Time Monitoring​

  • WebSocket Dashboard: Live test execution updates
  • Detailed Reporting: Performance metrics and error analysis
  • Historical Data: Track improvements over time

🏗️ Architecture​

graph LR
    subgraph "PunchFlow Testing"
        A[Test Dashboard]
        B[Universal Test Framework]
        C[Mock Buyer System]
        D[Shop API Testers]
    end
    
    subgraph "Target Shop"
        E[Shop Admin API]
        F[Shop Frontend]
        G[PunchOut Endpoint]
    end
    
    A --> B
    B --> C
    B --> D
    C --> G
    D --> E
    D --> F

🧪 Test Categories​

1. Authentication Tests​

Verify shop authentication mechanisms:

  • OAuth 2.0 flow validation
  • API key authentication
  • Token refresh handling
  • Permission verification

2. Product Catalog Tests​

Validate product data access:

  • Product listing retrieval
  • Search functionality
  • Category navigation
  • Price and availability checks

3. Cart Operations Tests​

Test cart manipulation:

  • Add/remove items
  • Quantity updates
  • Price calculations
  • Discount application

4. PunchOut Flow Tests​

Complete PunchOut scenarios:

  • Setup request handling
  • Session establishment
  • Cart transfer
  • Order message validation

5. Performance Tests​

Measure system performance:

  • Response time analysis
  • Concurrent session handling
  • Load testing
  • Stress testing

🚀 Quick Start​

1. Access Test Dashboard​

Navigate to the testing dashboard:

https://api.punchflow.de/api/v1/testing/dashboard

2. Configure Shop Connection​

{
    "shop_type": "shopware6",
    "shop_url": "https://demo.shopware.com",
    "credentials": {
        "client_id": "SWIACCESS123",
        "client_secret": "secret123",
        "auth_type": "oauth2"
    }
}

3. Select Test Suite​

Choose from predefined test suites or create custom ones:

  • Quick Test: Basic connectivity and authentication
  • Standard Test: Full functional validation
  • Performance Test: Load and stress testing
  • Custom Test: Select specific test cases

4. Execute Tests​

Click "Run Tests" and monitor real-time progress via WebSocket updates.

📊 Test Results​

Report Structure​

{
    "test_id": "test_2024_01_15_001",
    "shop": "demo.shopware.com",
    "timestamp": "2024-01-15T10:30:00Z",
    "duration": 45.2,
    "summary": {
        "total": 25,
        "passed": 23,
        "failed": 1,
        "skipped": 1
    },
    "categories": {
        "authentication": {
            "passed": 5,
            "failed": 0
        },
        "catalog": {
            "passed": 8,
            "failed": 0
        },
        "cart": {
            "passed": 6,
            "failed": 1
        },
        "punchout": {
            "passed": 4,
            "failed": 0
        }
    },
    "performance": {
        "avg_response_time": 234,
        "p95_response_time": 456,
        "p99_response_time": 789
    },
    "details": [...]
}

Understanding Results​

  • ✅ Passed: Test completed successfully
  • ❌ Failed: Test encountered an error
  • ⏭️ Skipped: Test was not applicable or dependency failed
  • ⚠️ Warning: Test passed with non-critical issues

🛠️ Advanced Features​

Mock Buyer Systems​

Simulate real procurement platforms:

# Configure mock buyer
mock_buyer = MockBuyerSystem(
    system_type="sap_ariba",
    network_id="AN12345",
    shared_secret="secret",
    return_url="https://buyer.example.com/return"
)

# Generate realistic PunchOut request
setup_request = mock_buyer.generate_punchout_setup_request(
    operation="create",
    buyer_cookie="SESSION123"
)

Custom Test Cases​

Create shop-specific test scenarios:

class CustomShopwareTest(UniversalShopTester):
    async def test_custom_workflow(self):
        # Your custom test logic
        result = await self.api_client.get("/custom-endpoint")
        assert result.status_code == 200
        return TestResult(
            name="Custom Workflow",
            status="passed",
            message="Custom workflow validated"
        )

Automated Testing​

Schedule regular test runs:

# celerybeat-schedule.py
CELERYBEAT_SCHEDULE = {
    'hourly-shop-tests': {
        'task': 'app.tasks.run_shop_tests',
        'schedule': crontab(minute=0),
        'args': (['shopware6', 'woocommerce'],)
    },
}

📈 Integration with Monitoring​

Test results are automatically exported to monitoring systems:

Prometheus Metrics​

  • punchout_test_duration_seconds: Test execution time
  • punchout_test_success_rate: Success rate per shop
  • punchout_test_failures_total: Total test failures

Grafana Dashboards​

Access test analytics at:

https://monitoring.punchflow.de/d/punchout-testing

Alerts​

Configure alerts for test failures:

- alert: ShopTestFailure
  expr: punchout_test_success_rate < 0.95
  for: 5m
  annotations:
    summary: "Shop {{ $labels.shop }} test success rate below 95%"

🔍 Troubleshooting​

Common Issues​

Authentication Failures​

Error: Authentication failed for shop
Solution: Verify API credentials and permissions

Network Timeouts​

Error: Request timeout after 30 seconds
Solution: Check shop availability and firewall rules

Invalid API Responses​

Error: Unexpected API response format
Solution: Verify shop version compatibility

Debug Mode​

Enable detailed logging:

tester = Shopware6ApiTester(
    shop_url="https://shop.example.com",
    debug=True,
    log_level="DEBUG"
)

📚 Next Steps​

Need help? Contact support@punchflow.de or visit our documentation.

Last updated on by Jannih