Skip to main content

Retry Delay Quickstart

Prerequisites​

Before using retry delay strategies, you should be familiar with:

Overview​

This quickstart guide helps you get started with retry delay strategies in NPipeline. Retry delays are essential for building resilient pipelines that can recover from transient failures without overwhelming downstream services.

🚀 Quick Start: Common Retry Patterns​

Web API Calls (Most Common)​

// Recommended for external API calls
context.UseExponentialBackoffDelay(
    baseDelay: TimeSpan.FromSeconds(1),
    multiplier: 2.0,
    maxDelay: TimeSpan.FromMinutes(1));

Why this pattern:

  • Exponential backoff gives services time to recover
  • Prevents thundering herd problems
  • Balanced for typical API rate limits

Database Operations​

// Recommended for database retry scenarios
context.UseLinearBackoffDelay(
    baseDelay: TimeSpan.FromMilliseconds(100),
    increment: TimeSpan.FromMilliseconds(200),
    maxDelay: TimeSpan.FromSeconds(5));

Why this pattern:

  • Linear growth is predictable for database connection pools
  • Shorter delays work well for transient lock contention
  • Conservative max delay prevents long-running transactions

File Processing​

// Recommended for file system operations
context.UseFixedDelay(TimeSpan.FromSeconds(2));

Why this pattern:

  • File system recovery is typically immediate
  • Fixed delay provides predictable behavior
  • Simple and effective for I/O operations

Decision Flow for Choosing Retry Strategy​

Implementation Examples​

Basic Pipeline with Retry Delays​

using NPipeline;
using NPipeline.Pipeline;

public sealed class RetryQuickstartPipeline : IPipelineDefinition
{
    public void Define(PipelineBuilder builder, PipelineContext context)
    {
        // Configure retry delay strategy based on operation type
        context.UseExponentialBackoffDelay(
            baseDelay: TimeSpan.FromSeconds(1),
            multiplier: 2.0,
            maxDelay: TimeSpan.FromMinutes(1));

        var source = builder.AddSource<ApiSource, ApiResponse>("api-source");
        var transform = builder.AddTransform<DataTransform, ApiResponse, ProcessedData>("transform");
        var sink = builder.AddSink<DataSink, ProcessedData>("sink");

        builder.Connect(source, transform);
        builder.Connect(transform, sink);

        // Configure retry options
        builder.WithRetryOptions(new PipelineRetryOptions(
            MaxItemRetries: 3,
            MaxNodeRestartAttempts: 2,
            MaxSequentialNodeAttempts: 5
        ));
    }
}

Per-Node Retry Configuration​

public void Define(PipelineBuilder builder, PipelineContext context)
{
    var source = builder.AddSource<DataSource, string>("source");
    
    // Fast retries for in-memory operations
    context.UseFixedDelay(TimeSpan.FromMilliseconds(50));
    var memoryTransform = builder.AddTransform<MemoryTransform, string, string>("memory-ops");
    
    // Slower retries for external API calls
    context.UseExponentialBackoffDelay(
        baseDelay: TimeSpan.FromSeconds(1),
        multiplier: 2.0,
        maxDelay: TimeSpan.FromMinutes(1));
    var apiTransform = builder.AddTransform<ApiTransform, string, string>("api-ops");
    
    // Default retries for other operations
    var sink = builder.AddSink<DataSink, string>("sink");

    builder.Connect(source, memoryTransform);
    builder.Connect(memoryTransform, apiTransform);
    builder.Connect(apiTransform, sink);
}

Common Pitfalls to Avoid​

⚠️ Common Mistake: Not configuring retry delays at all

Without retry delays, failed operations retry immediately, which can overwhelm recovering services.

⚠️ Common Mistake: Using exponential backoff for in-memory operations

In-memory operations typically recover immediately, so exponential backoff adds unnecessary delays.

⚠️ Common Mistake: Setting max delay too high

Very high max delays can cause long recovery times. Consider your SLA requirements.

💡 Pro Tip: Always test retry behavior in development

Use fixed delays in tests for predictable behavior, then switch to appropriate backoff in production.

Next Steps​

See Also​