↓ Skip to main content

VxWorks Interrupt Affinity on T1040: SMP Load Balancing

·5304 words·26 mins
VxWorks Interrupt Affinity T1040 Smp PowerPC MPIC Interrupt Routing Load Balancing Real-Time Systems Embedded-Systems

VxWorks Interrupt Affinity on T1040: SMP Load Balancing

📋 Abstract
#

Interrupt handling is a critical component of real-time embedded systems, particularly in airborne integrated core processors that communicate with radar, navigation, electronic warfare, infrared search, and other time-sensitive subsystems.

As embedded applications become increasingly complex, multi-core processors must handle growing interrupt workloads while maintaining predictable response times.

Conventional interrupt configurations often concentrate interrupt processing on a single processor core, creating a potential performance bottleneck even when other cores have available processing capacity.

This article investigates VxWorks interrupt affinity on the NXP QorIQ T1040 quad-core processor, focusing on the interaction between the Multi-Processor Interrupt Controller (MPIC) and the VxWorks Symmetric Multi-Processing (SMP) kernel.

After examining the T1040 interrupt-routing architecture and VxWorks interrupt-controller interfaces, a dynamic interrupt-affinity method based on processor load balancing is presented.

The method periodically measures interrupt activity, calculates per-core interrupt loads, and redistributes selected interrupt sources among available processor cores.

Experimental results from a T1040 platform running VxWorks 6.9 SMP show that dynamic interrupt routing improves interrupt-load distribution and reduces measured interrupt response latency under the tested workload.

For 100,000 interrupt stimuli, average response latency decreased from 3.795 µs to 3.489 µs, representing an improvement of approximately 8.1%.

The results demonstrate the potential of interrupt-affinity optimization for improving multi-core utilization and real-time responsiveness without modifying the underlying hardware.

Keywords: VxWorks interrupt affinity, T1040, MPIC, SMP, dynamic interrupt routing, interrupt load balancing, PowerPC, interrupt latency, airborne embedded systems.

🎯 1. Introduction to VxWorks Interrupt Affinity
#

1.1 Real-Time Requirements in Airborne Embedded Systems
#

Modern airborne integrated core processors connect multiple mission-critical subsystems, including:

  • Radar processing equipment.
  • Navigation and positioning systems.
  • Electronic warfare devices.
  • Infrared search and tracking systems.
  • Communication interfaces.
  • Sensor acquisition modules.
  • Flight management and control equipment.

These subsystems exchange information through hardware interfaces that frequently rely on interrupt-driven communication.

When an external device generates an interrupt, the processor must recognize the event, execute the appropriate interrupt service routine (ISR), and coordinate any subsequent data processing.

The performance of this mechanism directly affects system responsiveness.

Important requirements include:

  1. Low interrupt response latency.
  2. Predictable worst-case interrupt handling time.
  3. Efficient utilization of multiple processor cores.
  4. Minimal interference between interrupt processing and application tasks.
  5. Stable operation under high interrupt frequency.
  6. Deterministic handling of critical interrupt sources.

In systems with strict timing constraints, an overloaded interrupt-processing core can become a significant bottleneck.

1.2 Limitations of Single-Core Interrupt Routing
#

On a multi-core processor, interrupt sources may initially be routed to a designated boot processor.

For example, a four-core system might have the following configuration:

Processor Core Interrupt Processing Role
CPU0 Handles most external interrupts
CPU1 Primarily executes application tasks
CPU2 Primarily executes application tasks
CPU3 Primarily executes application tasks

Although the processor provides four execution cores, concentrating interrupt handling on CPU0 limits the potential benefits of parallel execution.

Under heavy interrupt activity, CPU0 may experience:

  • Increased interrupt response latency.
  • More frequent interruption of application tasks.
  • Higher interrupt-processing utilization.
  • Greater competition among interrupt sources.
  • Reduced responsiveness to lower-priority events.

Meanwhile, other cores may have sufficient processing capacity to handle additional interrupts.

This imbalance motivates the use of interrupt affinity.

1.3 What Is Interrupt Affinity?
#

Interrupt affinity is the mechanism that controls which processor core or set of cores is eligible to service a particular hardware interrupt.

In a multi-core operating system, interrupt affinity allows the software to distribute interrupt sources according to processing requirements.

Two approaches are commonly used.

Static Interrupt Affinity

Each interrupt source is assigned to a predetermined processor core.

The assignment generally remains unchanged during normal operation.

Dynamic Interrupt Affinity

Interrupt destinations are adjusted at runtime according to changing conditions such as processor utilization, interrupt frequency, and system workload.

Dynamic affinity is particularly useful when interrupt traffic varies significantly over time.

However, interrupt distribution must be coordinated with task scheduling, interrupt priorities, cache locality, and hardware routing constraints.

For related optimization techniques, see How to Optimize Interrupt Latency in VxWorks, which discusses ISR optimization, interrupt priorities, processor affinity, and real-time performance analysis.

1.4 Research Objectives
#

The proposed design focuses on three objectives:

  1. Analyze the interrupt-routing capabilities of the T1040 MPIC.
  2. Examine how VxWorks SMP manages processor-specific interrupt destinations.
  3. Develop and evaluate a dynamic interrupt-routing algorithm based on interrupt load.

The central design principle is to redistribute interrupt processing without changing the hardware configuration or application-level interrupt semantics.

🏗️ 2. NXP QorIQ T1040 Multi-Core Architecture
#

2.1 Overview of the T1040 Processor
#

The NXP QorIQ T1040 is a multi-core communications processor based on the Power Architecture.

The processor integrates four e5500 cores and provides hardware resources suitable for embedded networking, communications, and real-time data processing.

The experimental platform described in this study operates at 1.5 GHz and runs a VxWorks 6.9 SMP kernel.

Its relevant architectural features include:

  • Four Power Architecture processor cores.
  • Shared memory and multi-core execution support.
  • Hardware interrupt prioritization and routing.
  • Multi-Processor Interrupt Controller.
  • Inter-processor interrupt capabilities.
  • Integrated communication and peripheral controllers.
  • Software-configurable interrupt destination selection.

These features provide the hardware foundation required for processor-specific interrupt management.

2.2 Multi-Processor Interrupt Controller
#

The T1040 uses an MPIC to manage hardware interrupts and distribute them among processor cores.

The interrupt controller is responsible for:

  • Receiving interrupt requests from hardware sources.
  • Assigning interrupt vectors.
  • Managing interrupt priorities.
  • Selecting destination processors.
  • Delivering interrupts to eligible processor cores.
  • Supporting interrupt acknowledgment and completion.
  • Coordinating inter-processor interrupts.

Relevant interrupt categories include:

Interrupt Type Description
External interrupts Interrupt requests originating from external hardware signals
Internal interrupts Interrupts generated by integrated peripheral controllers
Inter-processor interrupts (IPI) Software-triggered notifications between processor cores
Message-signaled interrupts (MSI) Interrupt notifications associated with supported message-based mechanisms
Timer interrupts Interrupts generated by hardware timer facilities

This study focuses on external interrupt sources, with the source platform identifying external interrupt inputs IRQ[0:11].

The exact relationship between external interrupt pins, MPIC source indices, and VxWorks interrupt vectors depends on the processor documentation and BSP configuration.

2.3 Interrupt Routing Architecture
#

The interrupt-routing mechanism can be represented as follows:

+--------------------------------------+
|        Hardware Interrupts           |
|                                      |
| IRQ0  IRQ1  IRQ2  ...  IRQ11         |
+-------------------+------------------+
                    |
                    v
+--------------------------------------+
|        T1040 MPIC Controller         |
|                                      |
|  Interrupt Source Identification     |
|  Vector and Priority Management      |
|  Destination Processor Selection     |
+-------------------+------------------+
                    |
      +-------------+-------------+
      |             |             |
      v             v             v
   +------+      +------+      +------+
   | CPU0 |      | CPU1 |      | CPU2 |
   +------+      +------+      +------+
                                  |
                                  v
                               +------+
                               | CPU3 |
                               +------+

Each interrupt source has associated configuration information controlling how the MPIC processes and delivers it.

The destination configuration enables software to influence which CPU receives an interrupt.

🔧 3. T1040 MPIC Interrupt Affinity Mechanism
#

3.1 Interrupt Source Configuration
#

The MPIC maintains configuration information for individual interrupt sources.

Important properties include:

  • Interrupt vector.
  • Interrupt priority.
  • Interrupt enable or mask state.
  • Trigger characteristics.
  • Destination processor selection.

The processor destination information is especially important for implementing interrupt affinity.

3.2 External Interrupt Destination Register
#

In the MPIC architecture, an External Interrupt Destination Register (EIDR) controls the processor destinations associated with an interrupt source.

The register is 32 bits wide.

For the four-core configuration considered here, destination selection is represented conceptually by the lower four processor destination bits:

Destination Bit Processor
Bit 0 CPU0
Bit 1 CPU1
Bit 2 CPU2
Bit 3 CPU3

A one-hot destination mask assigns an interrupt to exactly one processor.

For example:

Destination CPU Destination Mask
CPU0 0x00000001
CPU1 0x00000002
CPU2 0x00000004
CPU3 0x00000008

A simple mask-generation operation is:

destinationMask = 1U << cpuId;

For cpuId = 2, the resulting mask is:

destinationMask = 0x00000004;

This selects CPU2 as the intended destination.

3.3 Single-Core Routing Versus Multiple Destinations
#

The proposed dynamic routing algorithm uses a single destination processor per interrupt source.

This simplifies interrupt accounting and makes assignment decisions deterministic.

However, the MPIC destination register should not be interpreted as universally restricting hardware configurations to exactly one destination bit.

Some MPIC implementations support destination masks containing multiple eligible processors, with delivery behavior depending on controller capabilities and operating mode.

The exact semantics must be verified against the T1040 reference manual and BSP implementation.

For the design presented here, one-hot destination masks are used to ensure that each managed interrupt source has a clearly defined processor owner.

3.4 Default Interrupt Routing
#

In the evaluated VxWorks configuration, external interrupts are initially associated with the boot processor, normally CPU0.

A simplified startup configuration is:

IRQ0  ---> CPU0
IRQ1  ---> CPU0
IRQ2  ---> CPU0
IRQ3  ---> CPU0
...
IRQ11 ---> CPU0

This arrangement simplifies initial interrupt-controller configuration.

However, when CPU0 receives a substantial number of high-frequency interrupts, the other cores may remain underutilized.

Interrupt affinity allows the system to change this distribution.

3.5 Interrupt Affinity Configuration Example
#

The following C code illustrates how a destination mask can be constructed for a four-core system.

#include <vxWorks.h>
#include <stdint.h>

#define T1040_CPU_COUNT  4U

STATUS mpicDestinationMaskGet
    (
    unsigned int cpuId,
    uint32_t *pMask
    )
{
    if (pMask == NULL)
        return ERROR;

    if (cpuId >= T1040_CPU_COUNT)
        return ERROR;

    *pMask = (uint32_t)(1U << cpuId);

    return OK;
}

This function validates the requested CPU and calculates the corresponding destination mask.

It does not directly program the MPIC.

Actual hardware configuration should be performed through the VxWorks interrupt-controller driver or an appropriate BSP-supported interface.

Direct register updates must account for interrupt synchronization, masking, pending interrupt state, and processor-specific register semantics.

⚙️ 4. VxWorks SMP Interrupt Affinity Implementation
#

4.1 VxWorks Symmetric Multi-Processing
#

Under VxWorks SMP, multiple processor cores execute within a shared operating-system instance.

The SMP kernel coordinates:

  • Multi-core task scheduling.
  • Interrupt handling.
  • Inter-processor communication.
  • Shared-memory synchronization.
  • CPU affinity.
  • Interrupt-controller initialization.
  • Processor startup and management.

The relevant build configuration uses the SMP-enabled kernel, commonly identified by:

_WRS_CONFIG_SMP

The availability of particular interrupt-routing APIs and internal structures depends on the VxWorks version and BSP.

4.2 Interrupt Controller CPU Mapping
#

The source implementation describes an interrupt-controller mapping structure:

struct intrCtrlCpus
{
    int inputPin;
    int cpuNum;
};

This structure associates an interrupt input with a processor number.

Its purpose is to provide the controller driver with information needed to select the destination processor.

The exact declaration and use of this structure should be verified against the VxWorks 6.9 source code for the selected BSP.

It is not a general portable VxWorks application API.

4.3 MPIC Interrupt Initialization
#

The source implementation identifies the following internal controller function:

mpicIntCtrlIntInit()

During initialization, the interrupt-controller driver configures the initial interrupt routing state.

Typical responsibilities include:

  1. Initializing MPIC control registers.
  2. Establishing interrupt vector and priority configuration.
  3. Configuring destination processor information.
  4. Connecting interrupt sources with the kernel’s interrupt infrastructure.
  5. Enabling interrupts when their handlers and controller configuration are ready.

In the reported T1040 BSP, managed interrupts are initially directed to the boot processor.

Interrupt masking should follow the BSP’s initialization sequence. It is unsafe to assume that all interrupt sources can be unmasked unconditionally during startup.

4.4 Runtime Interrupt Rerouting
#

The source implementation identifies the following function as a key runtime mechanism:

mpicIntCtrlIntReroute()

This controller-specific routine updates the target processor for an interrupt source.

Its conceptual workflow is:

  1. Validate the requested processor number.
  2. Translate the logical CPU identifier to the controller’s hardware destination representation.
  3. Resolve the interrupt source or input pin.
  4. Obtain the interrupt’s configuration state.
  5. Update the destination information.
  6. Apply the controller-specific synchronization required for safe routing.
  7. Restore the appropriate interrupt enable state.

The implementation may use register-access macros such as:

MPIC_REG_WRITE(...);

The actual function signature, register offsets, and supported routing modes depend on the installed VxWorks controller driver.

They should be obtained from the BSP source rather than inferred from the function name.

4.5 Dynamic Affinity Through VxBus
#

In VxBus-based VxWorks configurations, interrupt-management interfaces may support routing operations through device-oriented APIs.

Examples described in the VxBus interrupt architecture include:

  • vxbIntToCpuRoute()
  • vxbIntReroute()

These interfaces abstract portions of the interrupt-controller implementation.

For a broader explanation of SMP interrupt management, see VxBus Architecture and Driver Model for VxWorks SMP.

The article describes VxBus interrupt routing, driver registration, interrupt-controller integration, and synchronization considerations in multi-core systems.

Application developers should prefer supported interrupt-management interfaces where available, rather than modifying MPIC registers directly.

4.6 Runtime Rerouting Considerations
#

Changing an interrupt’s destination does not necessarily migrate an interrupt that is already being serviced.

Pending, active, and subsequently generated interrupts may be handled differently depending on controller semantics.

A safe routing implementation should consider:

  • Whether the interrupt is enabled.
  • Whether an interrupt is currently active.
  • Whether the source is level-triggered or edge-triggered.
  • Whether pending events are preserved.
  • Whether the destination CPU is online.
  • Whether the ISR is safe to execute on another processor.
  • Whether associated deferred-processing tasks have processor-affinity constraints.

The destination update generally affects eligible future interrupt deliveries, subject to the MPIC’s documented routing behavior.

🔀 5. Static and Dynamic Interrupt Routing Strategies
#

5.1 Static Interrupt Affinity
#

Static affinity assigns interrupt sources to fixed processor cores.

An example distribution is:

Interrupt Source Assigned CPU
IRQ0 CPU0
IRQ1 CPU1
IRQ2 CPU2
IRQ3 CPU3
IRQ4 CPU0
IRQ5 CPU1
IRQ6 CPU2
IRQ7 CPU3

Advantages include:

  • Simple configuration.
  • Predictable interrupt placement.
  • Minimal runtime routing overhead.
  • Improved control over processor resource allocation.
  • Easier analysis of critical interrupt paths.

However, static affinity cannot automatically adapt when interrupt frequency changes.

A core may become overloaded even when the initial configuration appears balanced.

5.2 Dynamic Interrupt Affinity
#

Dynamic affinity periodically evaluates interrupt activity and adjusts processor assignments.

The proposed design uses a dedicated monitoring task to collect interrupt statistics and update MPIC destination configurations.

The architecture is:

+-----------------------------------+
|       Interrupt Statistics        |
|                                   |
| Per-IRQ Counts for Each CPU       |
+-----------------+-----------------+
                  |
                  v
+-----------------------------------+
|       Load Calculation            |
|                                   |
| Weighted Per-Core Interrupt Load  |
+-----------------+-----------------+
                  |
                  v
+-----------------------------------+
|      Routing Decision Engine      |
|                                   |
| Interrupt Frequency Analysis      |
| Core Load Comparison              |
+-----------------+-----------------+
                  |
                  v
+-----------------------------------+
|       MPIC Affinity Update        |
|                                   |
| Reassign Selected Interrupts      |
+-----------------------------------+

This design separates monitoring, decision-making, and hardware configuration.

📊 6. Dynamic Interrupt Load-Balancing Algorithm
#

6.1 Interrupt Activity Sampling
#

The algorithm divides system execution into monitoring intervals of duration \(T\).

For example:

\[ T = 100\ \text{ms} \]

At the end of each interval, the monitoring task reads interrupt counters for each processor.

Let:

  • \(m\): Number of monitored interrupt sources.
  • \(c\): Number of available processor cores.
  • \(r\): Interrupt source index, \(0 \le r < m\).
  • \(i\): Processor index, \(0 \le i < c\).
  • \(n\): Current monitoring interval.
  • \(N_{r,i}(nT)\): Cumulative number of occurrences of interrupt \(r\) on CPU \(i\) at time \(nT\).

The number of interrupts observed during the current interval is:

\[ \Delta N_{r,i}(n) = N_{r,i}(nT) - N_{r,i}((n-1)T) \]

These interval counts provide the basis for estimating interrupt load.

6.2 Weighted Processor Interrupt Load
#

The source algorithm computes the weighted interrupt load for each processor.

Using \(w_r\) as the workload weight associated with interrupt source \(r\):

$$ L_i(n) = \sum_{r=0}^{m-1} \Delta N_{r,i}(n)\cdot w_r $$

Where:

  • \(L_i(n)\) is the estimated interrupt load of CPU \(i\).
  • \(\Delta N_{r,i}(n)\) is the number of interrupt occurrences during the sampling interval.
  • \(w_r\) represents the relative cost or importance assigned to interrupt \(r\).

The original formulation uses interrupt priority as the weighting factor.

This can be useful as a scheduling heuristic, but hardware priority values do not necessarily represent CPU execution cost.

Depending on the interrupt controller, larger numeric values may not even correspond to higher priority.

For more accurate load estimation, the weight should preferably represent measured interrupt service time or another calibrated processing-cost metric.

6.3 Interrupt Occurrence Frequency
#

The occurrence count of each interrupt across all processor cores is:

$$ M_r(n) = \sum_{i=0}^{c-1} \Delta N_{r,i}(n) $$

The estimated interrupt frequency is:

$$ F_r(n) = \frac{M_r(n)}{T} $$

Where \(F_r(n)\) is measured in interrupts per second.

For example, if interrupt IRQ5 occurs 500 times during a 100 ms interval:

$$ F_5 = \frac{500}{0.1} = 5000\ \text{interrupts/s} $$

This information identifies high-frequency interrupt sources that may contribute disproportionately to processor load.

6.4 Greedy Load-Balancing Strategy
#

The original routing algorithm performs the following operations:

  1. Collect interrupt occurrence counts.
  2. Calculate the weighted load of each processor core.
  3. Calculate the occurrence frequency of each interrupt.
  4. Sort interrupts by descending occurrence frequency.
  5. Sort processor cores by ascending interrupt load.
  6. Assign high-frequency interrupts to relatively lightly loaded cores.
  7. Update the MPIC destination registers.

A practical refinement is to update the projected processor load after each assignment.

Without this update, repeatedly assigning interrupts using only the original load ranking can produce an imbalanced result.

The predicted processing demand of interrupt \(r\) can be estimated as:

\[ C_r = M_r \cdot w_r \]

The load assigned to CPU \(i\) after adding interrupt \(r\) becomes:

$$ \widehat{L}_i^{\,new} = \widehat{L}_i^{\,old} + C_r $$

The next interrupt is assigned to the eligible CPU with the lowest projected load.

This produces a more meaningful greedy approximation of balanced interrupt processing.

6.5 Algorithm Pseudocode
#

The following pseudocode illustrates the load-balancing procedure:

/*
 * Dynamic interrupt affinity algorithm.
 *
 * Inputs:
 *   irqCount[r][i]  - Interval interrupt counts
 *   irqWeight[r]    - Estimated processing cost
 *   irqCountTotal   - Number of managed IRQs
 *   cpuCount        - Number of available CPUs
 *
 * Output:
 *   targetCpu[r]    - Proposed destination CPU
 */

for each CPU i:
    baseLoad[i] = estimateNonMigratableLoad(i)
    projectedLoad[i] = baseLoad[i]

for each interrupt r:
    irqFrequency[r] = 0

    for each CPU i:
        irqFrequency[r] += irqCount[r][i]

    irqCost[r] =
        irqFrequency[r] * irqWeight[r]

sort interrupts by descending irqCost

for each interrupt r:

    if interruptIsPinned(r):
        targetCpu[r] = currentCpu[r]
        continue

    targetCpu[r] =
        findLeastLoadedEligibleCpu(projectedLoad, r)

    projectedLoad[targetCpu[r]] += irqCost[r]

for each interrupt r:

    if targetCpu[r] != currentCpu[r]:

        if rerouteAllowed(r, targetCpu[r]):
            rerouteInterrupt(r, targetCpu[r])

The pseudocode is an algorithmic description rather than directly compilable VxWorks source code.

A production implementation must provide:

  • Interrupt statistics collection.
  • Processor eligibility masks.
  • CPU availability checks.
  • Controller-specific rerouting operations.
  • Synchronization with active interrupt handlers.
  • Migration rate limiting.
  • Error handling and rollback.

6.6 Algorithm Time Complexity
#

Let:

  • \(m\) represent the number of interrupt sources.
  • \(c\) represent the number of processor cores.

Collecting and processing interrupt counts requires:

\[ O(mc) \]

Sorting interrupts requires:

\[ O(m\log m) \]

With a linear search for the least-loaded eligible processor during every assignment, processor selection requires:

\[ O(mc) \]

Therefore, the overall complexity is approximately:

\[ O(mc + m\log m) \]

For the T1040, the processor count is fixed at four.

Consequently, the computation remains relatively inexpensive for a moderate number of interrupt sources.

The original estimate of \(O(mc)\) describes the principal counting and assignment operations but does not include the sorting cost.

6.7 Selecting the Monitoring Interval
#

The monitoring interval determines how frequently the algorithm responds to changing interrupt workloads.

A shorter interval provides faster adaptation but increases monitoring and routing overhead.

A longer interval reduces overhead but may allow temporary load imbalances to persist.

The evaluated design uses:

\[ T = 100\ \text{ms} \]

Therefore, the algorithm executes:

\[ f = \frac{1}{T} = 10\ \text{Hz} \]

For a representative application cycle rate of 60 Hz, each monitoring interval covers approximately six application cycles.

However, a 60 Hz application workload does not necessarily imply that all interrupt sources operate at 60 Hz.

The monitoring period should be selected based on the actual interrupt arrival patterns, control-loop deadlines, and acceptable migration overhead.

6.8 Avoiding Excessive Interrupt Migration
#

Dynamic rerouting introduces costs beyond the load-balancing calculation itself.

Frequent migration may affect:

  • Cache locality.
  • Deferred ISR execution.
  • Interrupt-controller synchronization.
  • Processor load stability.
  • Worst-case interrupt response time.

A production implementation should use migration controls such as:

Minimum Load Improvement

Only migrate when the predicted reduction in imbalance exceeds a defined threshold.

Migration Cooldown

Prevent repeated reassignment of the same interrupt within a short interval.

Critical Interrupt Pinning

Keep timing-critical interrupt sources bound to designated cores.

Processor Eligibility Masks

Restrict interrupts to processor cores capable of servicing them correctly.

Load Smoothing

Use multiple monitoring intervals to reduce sensitivity to transient interrupt bursts.

These controls are especially important for safety-critical embedded applications.

🧪 7. Experimental Platform and Test Configuration
#

7.1 Hardware Configuration
#

The experimental platform described in the source study uses the following configuration:

Component Configuration
Processor NXP QorIQ T1040
Processor architecture Power Architecture
CPU cores 4
Operating frequency 1.5 GHz
Interrupt controller MPIC
Operating system VxWorks 6.9 SMP
Optimization Dynamic interrupt affinity
Monitoring interval 100 ms

The platform provides a practical environment for evaluating interrupt distribution and response latency under a multi-core RTOS.

7.2 Test Methodology
#

The evaluation compares two configurations.

Baseline Configuration

Interrupt routing follows the original BSP configuration, with most monitored external interrupts directed to CPU0.

Optimized Configuration

The dynamic load-balancing task periodically evaluates interrupt activity and updates destination processor assignments.

The experimental comparison examines:

  1. Distribution of interrupt counts across cores.
  2. Response latency of representative interrupts.
  3. Maximum, minimum, and average interrupt response times.
  4. Algorithm execution overhead.

The reported measurements are retained from the supplied study. Raw traces, variance estimates, and test instrumentation details were not provided, so the results should be interpreted within the scope of the reported experiment.

📈 8. Experimental Results: Interrupt Load Distribution
#

8.1 Baseline Interrupt Distribution
#

In the baseline configuration, nearly all monitored external interrupts were handled by CPU0.

The resulting system showed an uneven interrupt-processing distribution.

Conceptually:

Baseline Interrupt Distribution

CPU0  ############################
CPU1  #
CPU2  #
CPU3  #

# represents relative interrupt activity

CPU0 handled most of the interrupt workload, while other processors had considerably less interrupt activity.

This concentration increased the opportunity for interrupt contention on the boot processor.

8.2 Dynamic Routing Distribution
#

After enabling dynamic interrupt routing, high-frequency interrupt sources were reassigned to other processor cores.

The resulting distribution was more balanced.

Conceptually:

Dynamic Interrupt Distribution

CPU0  ########
CPU1  #######
CPU2  #########
CPU3  ########

# represents relative interrupt activity

The conceptual diagrams illustrate the reported trend rather than reproducing measured interrupt counts.

The original study indicates that interrupt distribution curves became more uniform after the optimization.

This suggests that the routing mechanism successfully reduced the concentration of interrupt activity on CPU0.

8.3 Interpretation of Load Balancing Results
#

Improved distribution of interrupt counts is an important result, but equal interrupt counts do not necessarily imply equal processor utilization.

Different interrupt handlers may have significantly different execution times.

For example:

  • One interrupt may require only a short hardware acknowledgment.
  • Another may perform register reads and buffer management.
  • A third may activate substantial deferred processing.

Therefore, the most useful load-balancing metric is often interrupt-related CPU execution time, rather than interrupt frequency alone.

The proposed weighted algorithm provides a foundation for incorporating this distinction.

⏱️ 9. Interrupt Response Latency Analysis
#

9.1 Representative Interrupt Measurements
#

The experiment evaluated response times for six representative interrupt sources.

The source study reports that high-frequency interrupt vectors, including IRQ24, IRQ25, and IRQ41, experienced notable improvements after dynamic routing was enabled.

Lower-frequency interrupts showed smaller improvements or slight performance degradation.

This behavior is consistent with the expected tradeoff.

High-frequency sources can benefit from reduced contention on an overloaded core.

Low-frequency sources may experience relatively little benefit while still being affected by monitoring activity and routing overhead.

The IRQ numbers here represent the identifiers used by the experimental software and should not automatically be interpreted as physical IRQ[0:11] input-pin numbers.

9.2 Response Time Comparison
#

A further experiment applied 100,000 equivalent interrupt stimuli to the baseline and dynamically routed configurations.

The reported response-time statistics were:

Configuration Maximum (µs) Minimum (µs) Average (µs)
Default routing 3.855 3.209 3.795
Dynamic routing 3.600 2.715 3.489

Dynamic interrupt routing reduced all three reported latency metrics.

9.3 Maximum Response Time Improvement
#

The baseline maximum response time was:

\[ T_{\max,default}=3.855\ \mu s \]

After optimization:

\[ T_{\max,dynamic}=3.600\ \mu s \]

The improvement is:

\[ \frac{3.855-3.600}{3.855}\times100\% \approx6.6\% \]

Thus, the measured maximum response time decreased by approximately 6.6%.

9.4 Minimum Response Time Improvement
#

The baseline minimum response time was:

\[ T_{\min,default}=3.209\ \mu s \]

After optimization:

\[ T_{\min,dynamic}=2.715\ \mu s \]

The improvement is:

\[ \frac{3.209-2.715}{3.209}\times100\% \approx15.4\% \]

The measured minimum response time decreased by approximately 15.4%.

9.5 Average Response Time Improvement
#

The baseline average response time was:

\[ T_{avg,default}=3.795\ \mu s \]

After optimization:

\[ T_{avg,dynamic}=3.489\ \mu s \]

The improvement is:

$$ \frac{3.795-3.489}{3.795}\times100\% \approx8.1\% $$

The average response time decreased by approximately 8.1%.

The source study describes this improvement as approximately 10%; the more precise value calculated from the supplied measurements is 8.1%.

9.6 Summary of Measured Improvements
#

Performance Metric Default Dynamic Improvement
Maximum latency 3.855 µs 3.600 µs 6.6%
Minimum latency 3.209 µs 2.715 µs 15.4%
Average latency 3.795 µs 3.489 µs 8.1%

These results show a reduction in the measured interrupt response latency under the tested conditions.

However, maximum, minimum, and average values alone are insufficient to establish the complete latency distribution.

A stronger real-time analysis would also include percentiles, standard deviation, measurement methodology, and worst-case behavior under sustained stress.

⚡ 10. Dynamic Routing Algorithm Overhead
#

10.1 Micro-Benchmark Results
#

The source study reports a micro-benchmark in which the routing algorithm was executed one million times.

Total execution time:

\[ T_{total}=899\ \text{ms} \]

Number of executions:

\[ N=1{,}000{,}000 \]

Average execution time:

$$ T_{execution} = \frac{899\ \text{ms}}{1{,}000{,}000} $$

Therefore:

\[ T_{execution}=0.899\ \mu s \]

The average measured computational cost was approximately 0.9 µs per invocation.

10.2 Monitoring Overhead at 100 ms Intervals
#

With a monitoring interval of 100 ms, the routing algorithm executes ten times per second.

The estimated computational cost is:

$$ T_{overhead} = 10\times0.899\ \mu s $$

Thus:

\[ T_{overhead}=8.99\ \mu s/s \]

This represents approximately:

\[ 0.000899\% \]

of one processor core’s available execution time.

The measured computation overhead is therefore very small.

However, this figure represents the reported algorithm micro-benchmark, not necessarily the complete runtime cost of dynamic interrupt affinity.

Additional overhead may include:

  • Interrupt counter collection.
  • Cache and synchronization effects.
  • Sorting and allocation operations.
  • Task scheduling.
  • Lock acquisition.
  • MPIC register updates.
  • Interrupt masking and rerouting.
  • Migration-related cache locality changes.

A complete performance assessment should measure the full monitoring and rerouting path on the target platform.

10.3 Computational Efficiency Versus Real-Time Predictability
#

Low average execution overhead does not automatically guarantee deterministic behavior.

A load-balancing task may interfere with time-critical processing if it executes at an inappropriate priority or modifies interrupt routing during a sensitive operation.

For hard real-time systems, the monitoring mechanism should:

  • Execute at a controlled priority.
  • Have bounded execution time.
  • Avoid blocking critical ISRs.
  • Minimize shared-resource contention.
  • Restrict migration of critical interrupt sources.
  • Maintain deterministic behavior when rerouting fails.

In practice, predictability may be more important than achieving perfectly uniform interrupt distribution.

🛡️ 11. Engineering Considerations for Production Systems
#

11.1 Interrupt Priority Must Be Preserved
#

Dynamic affinity controls where an interrupt is delivered, not necessarily its hardware priority.

Interrupt priority and processor affinity are separate configuration dimensions.

A routing algorithm must avoid inadvertently changing a critical interrupt’s priority.

For airborne systems, high-priority interrupts associated with safety-critical functions may require dedicated processor assignments.

11.2 Interrupt Affinity and Task Affinity Must Be Coordinated
#

Many ISRs perform minimal work and delegate processing to tasks.

If an interrupt is routed to CPU2 while its associated processing task is fixed to CPU0, the system may incur additional cross-core communication and cache traffic.

This can offset the expected improvement from interrupt balancing.

Where appropriate, designers should coordinate:

  • ISR processor affinity.
  • Deferred processing task affinity.
  • Application task placement.
  • Shared data locality.
  • Inter-processor communication overhead.

The optimal configuration may place an ISR and its primary processing task on the same core, although this depends on workload and scheduling requirements.

11.3 Synchronization Across Processor Cores
#

The monitoring task and interrupt-controller driver access shared state.

Examples include:

  • Per-core interrupt counters.
  • Current interrupt destinations.
  • Processor availability masks.
  • Routing statistics.
  • Migration control flags.

Shared data must be protected using SMP-safe synchronization mechanisms.

Depending on the context, suitable techniques may include:

  • Atomic operations.
  • Spinlocks.
  • Mutexes for task-context operations.
  • Memory barriers.
  • Per-core data structures.

Blocking synchronization primitives must not be used inside ISRs when their semantics prohibit blocking.

11.4 Avoid Interrupt Migration During Active Processing
#

Rerouting an interrupt while it is active may introduce controller-specific complications.

The routing layer should follow the documented MPIC and VxWorks rules for changing destinations.

Possible safeguards include:

  1. Validating the destination CPU.
  2. Coordinating with the interrupt-controller driver.
  3. Preventing conflicting concurrent routing updates.
  4. Preserving the source’s enable and priority configuration.
  5. Handling pending and active interrupts correctly.
  6. Verifying the destination change.
  7. Restoring the previous route if configuration fails.

These operations should be implemented through supported controller interfaces whenever possible.

11.5 Real-Time Certification Considerations
#

Airborne software may be subject to stringent safety and certification requirements.

A dynamic optimization mechanism introduces additional runtime behavior that must be analyzed.

Potential concerns include:

  • Non-deterministic interrupt migration.
  • Worst-case monitoring-task execution time.
  • Interference with critical processing.
  • Shared-resource contention.
  • Failure recovery.
  • Configuration verification.
  • Traceability of routing decisions.

For systems requiring strict partitioning or certification evidence, static interrupt affinity may be preferable for critical interrupt sources.

A hybrid design can retain fixed routing for safety-critical interrupts while dynamically balancing lower-criticality interrupts.

🔬 12. Limitations and Further Research
#

12.1 Scope of the Current Study
#

The proposed algorithm focuses on external interrupt routing on the T1040 platform.

It does not provide a unified load-balancing framework for every interrupt type.

Internal peripheral interrupts, timer interrupts, and inter-processor interrupts may have different routing restrictions.

These differences must be considered before extending the mechanism.

12.2 Improving the Load Estimation Model
#

The current algorithm estimates interrupt load using occurrence counts and weights.

A more accurate model could include measured ISR execution time.

For example:

$$ L_i = \sum_{r=0}^{m-1} \Delta N_{r,i}\cdot E_r $$

Where:

\[ E_r \]

represents the estimated execution time of interrupt handler \(r\).

This approach better approximates the CPU time consumed by interrupt processing.

Additional inputs could include:

  • Deferred interrupt-processing time.
  • Application task utilization.
  • Cache-miss behavior.
  • Processor affinity restrictions.
  • Interrupt criticality.
  • Inter-core communication cost.

12.3 Extending Interrupt Affinity to Other Sources
#

The same general framework could be extended to selected internal peripherals and interrupt types supported by the hardware and operating system.

Potential areas include:

  • Internal peripheral interrupts.
  • Timer-related interrupts.
  • Message-signaled interrupts.
  • Inter-processor communication.
  • Multi-core device-driver interrupt management.

However, some interrupt sources are inherently processor-local or must remain associated with specific cores.

A unified framework should respect these restrictions rather than treating all interrupt sources as freely migratable.

12.4 Strengthening Experimental Validation
#

The reported measurements demonstrate a useful performance improvement, but additional validation would strengthen the conclusions.

Future experimental analysis should include:

  • Full response-time distributions.
  • High-percentile latency measurements.
  • Worst-case response under competing workloads.
  • Interrupt migration frequency.
  • Per-core ISR execution time.
  • Load-balancing stability.
  • Monitoring and rerouting overhead.
  • Cache locality effects.
  • Behavior during simultaneous interrupt bursts.
  • Long-duration stress testing.

These measurements would help distinguish improvements in average throughput from improvements in worst-case real-time predictability.

✅ 13. Conclusion
#

Interrupt affinity is an important optimization mechanism for multi-core VxWorks systems.

On the NXP QorIQ T1040 processor, the MPIC provides hardware capabilities for routing interrupts to selected processor cores, while the VxWorks SMP interrupt-controller infrastructure enables software to manage these assignments.

Static affinity offers predictable interrupt placement but cannot automatically respond to changing workloads.

Dynamic interrupt affinity addresses this limitation by periodically collecting interrupt statistics, estimating processor load, and redistributing selected interrupt sources.

The proposed load-balancing method demonstrates several advantages:

  1. Improved Interrupt Distribution: High-frequency interrupt sources are distributed across multiple processor cores instead of remaining concentrated on CPU0.

  2. Reduced Interrupt Response Latency: The reported experimental data show reductions of approximately 6.6% in maximum latency, 15.4% in minimum latency, and 8.1% in average latency.

  3. Low Computational Overhead: The routing algorithm’s reported average execution time is approximately 0.899 µs per invocation.

  4. No Hardware Modification: The optimization relies on software-controlled MPIC destination configuration and existing processor resources.

  5. Adaptability: Interrupt assignments can respond to changing workload patterns without requiring a fixed routing configuration.

For production real-time systems, dynamic routing should also account for interrupt criticality, ISR execution time, task affinity, migration overhead, and worst-case timing behavior.

The central engineering lesson is that effective interrupt affinity is not simply about distributing interrupt counts evenly. It is about assigning interrupt-processing work to available processor cores while preserving deterministic response and system-level real-time constraints.

With appropriate synchronization, workload estimation, and timing validation, dynamic interrupt routing can improve the performance and scalability of VxWorks SMP systems used in airborne integrated processors and other demanding embedded applications.

Related