VxWorks 6.9 AMP: Multi-Core Image Builds and CPU Launch

VxWorks 6.9 AMP: Multi-Core Image Builds and CPU Launch

Asymmetric Multi-Processing (AMP) allows a multi-core processor to run multiple independent operating-system instances simultaneously. Instead of having one operating system control every CPU, each core can run its own image, application set, and memory region.

This makes AMP fundamentally different from Symmetric Multi-Processing (SMP), where a single operating-system instance manages multiple CPUs and generally provides a shared execution environment.

AMP is especially useful when a system needs strong functional separation, mixed operating systems, incremental migration of legacy applications, or isolation of safety- and security-sensitive workloads.

This guide focuses on unsupervised AMP with VxWorks 6.9, using the Freescale P4080 eight-core PowerPC processor as a practical example. The P4080 is a complex multi-core platform, and configuring all eight CPUs manually can quickly become tedious.

The goal is therefore not only to explain the AMP architecture, but also to show how to automate per-core image generation, load those images with wrload, and access secondary VxWorks shells through MIPC serial devices.

Scope: This article covers unsupervised AMP. Hypervisor-based or otherwise supervised AMP is outside the scope of this guide.

🧩 Understanding AMP on VxWorks

#

In an AMP system, each CPU can have its own private operating-system instance and memory allocation.

A simplified architecture looks like this:

+------------------------------------------------------+
|                  Multi-Core Processor                |
|                                                      |
|  CPU0          CPU1          CPU2          CPU3      |
|  VxWorks      VxWorks      VxWorks      VxWorks      |
|  Image 0      Image 1      Image 2      Image 3      |
|  Private      Private      Private      Private      |
|  Memory       Memory       Memory       Memory       |
|                                                      |
|  CPU4          CPU5          CPU6          CPU7      |
|  VxWorks      VxWorks      VxWorks      VxWorks      |
|  Image 4      Image 5      Image 6      Image 7      |
|  Private      Private      Private      Private      |
|  Memory       Memory       Memory       Memory       |
|                                                      |
|            Shared Memory / IPC Region                |
+------------------------------------------------------+

CPU0 normally performs the initial boot operation and can then load the images intended for the remaining CPUs.

The memory map is typically divided into private and shared regions.

Private Memory

#

Each CPU receives a region reserved for its own VxWorks image, kernel data, stacks, heap, and applications.

This provides a strong degree of logical isolation because an application running on one CPU does not normally need to use the memory assigned to another CPU.

CPU0 is often given more memory because it may perform additional management and networking functions.

Shared Memory

#

A separate shared-memory region can be used for inter-CPU communication.

Depending on the system architecture, that region can support mechanisms such as:

• Shared data structures
• Message passing
• MIPC communication
• Inter-processor signaling
• Shared buffers

This creates a useful separation:

Private memory  → local OS and application state
Shared memory   → controlled communication between CPUs

🏗️ Why Use AMP?

#

AMP is useful when a multi-core system needs independent software environments instead of a single shared operating-system instance.

Modular Application Design

#

Different functional domains can be assigned to different CPUs.

For example:

CPU0 → System management / control plane
CPU1 → Packet processing / data plane
CPU2 → DSP or signal-processing workload
CPU3 → Diagnostics

Each software domain can have its own VxWorks configuration and application image.

This can simplify architectural separation and reduce the need to combine unrelated software components into one operating-system image.

Security and Isolation

#

AMP can also be used to isolate sensitive workloads.

A secure application and its associated I/O can be assigned to a dedicated CPU and protected through the processor’s memory-management mechanisms.

The exact strength of isolation depends on the processor, BSP, memory configuration, and overall system design, but the basic architecture provides a useful boundary between workloads.

Mixed Operating Systems

#

One of AMP’s major advantages is the ability to run different operating-system instances on different CPUs.

For example:

CPU0 → VxWorks
CPU1 → VxWorks
CPU2 → Linux
CPU3 → Linux

This can be valuable when an existing Linux application needs to coexist with a deterministic VxWorks control workload.

Legacy System Consolidation

#

AMP is also useful when consolidating software that previously ran on several separate boards.

Instead of merging the legacy codebases into one operating-system image, individual applications can be assigned to separate CPUs on a multi-core processor.

Existing network-oriented communication paths can also be preserved where appropriate through mechanisms such as MIPC network devices.

This makes AMP a practical migration strategy when software consolidation is desired but a full rewrite is not.

⚖️ AMP Versus SMP

#

AMP and SMP solve different architectural problems.

AMP is therefore not simply an alternative implementation of SMP.

The two approaches reflect different system-level requirements.

🖥️ VxWorks 6.9 AMP on the P4080

#

The Freescale P4080 is an eight-core PowerPC processor that provides a useful example of AMP deployment.

Its multi-core architecture makes it possible to create a system where each CPU executes an independent VxWorks instance.

The basic startup model is:

                 Reset / Power On
                        |
                        v
                 +--------------+
                 |     CPU0     |
                 |   VxWorks    |
                 +------+-------+
                        |
                  wrload images
                        |
        +-------+-------+-------+-------+
        |       |       |       |       |
        v       v       v       v       v
      CPU1    CPU2    CPU3    CPU4   ... CPU7
      VxWorks VxWorks VxWorks VxWorks   VxWorks

CPU0 acts as the boot and management processor.

The wrload utility can then load the images for the other CPUs into their assigned memory regions and initiate their startup.

Because each CPU has a separate image, eight cores generally mean eight separately configured VxWorks projects.

That is exactly where automation becomes valuable.

🛠️ Building Per-Core VxWorks Images

#

There are two practical approaches for generating AMP images:

1. Use the VxWorks Workbench GUI for each CPU.
2. Automate the process with a Makefile.

For a system with eight CPUs, the GUI approach quickly becomes repetitive.

A command-line build system provides several advantages:

• Repeatable configuration
• Faster rebuilds
• Consistent component selection
• Easy integration with scripts and CI environments
• Clear mapping between CPU number and image
• Reduced manual configuration errors

The example below creates one project per CPU and copies the resulting vxWorks image into a common directory.

📦 Example AMP Makefile

#

The following example is based on an eight-core P4080 platform and illustrates the basic build pattern.

###############################################################################
#
# Makefile for VxWorks AMP builds
#
# Example target: AMC718 / Freescale P4080
#
# From a VxWorks development shell:
#
#   make amp
#
# The generated VxWorks images are copied to FTP_DIR.
#
###############################################################################

WIND_HOME := $(subst \,/,$(WIND_HOME))
WIND_BASE := $(subst \,/,$(WIND_BASE))
WIND_USR  := $(subst \,/,$(WIND_USR))

TOOL = gnu

# Directory used to collect generated images.
FTP_DIR = C:/temp/incoming

AMP0_DIR  = amc718_gnu_amp0
AMP0_PROJ = $(AMP0_DIR)/$(AMP0_DIR).wpj

AMP1_DIR  = amc718_gnu_amp1
AMP1_PROJ = $(AMP1_DIR)/$(AMP1_DIR).wpj

AMP2_DIR  = amc718_gnu_amp2
AMP2_PROJ = $(AMP2_DIR)/$(AMP2_DIR).wpj

AMP3_DIR  = amc718_gnu_amp3
AMP3_PROJ = $(AMP3_DIR)/$(AMP3_DIR).wpj

AMP4_DIR  = amc718_gnu_amp4
AMP4_PROJ = $(AMP4_DIR)/$(AMP4_DIR).wpj

AMP5_DIR  = amc718_gnu_amp5
AMP5_PROJ = $(AMP5_DIR)/$(AMP5_DIR).wpj

AMP6_DIR  = amc718_gnu_amp6
AMP6_PROJ = $(AMP6_DIR)/$(AMP6_DIR).wpj

AMP7_DIR  = amc718_gnu_amp7
AMP7_PROJ = $(AMP7_DIR)/$(AMP7_DIR).wpj


amp: amp0 amp1 amp2 amp3 amp4 amp5 amp6 amp7


amp0:
	vxprj create -force $(BSP) $(TOOL) $(AMP0_PROJ)
	vxprj bundle add $(AMP0_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP0_PROJ) BUNDLE_AMP_PRI
	vxprj component add $(AMP0_PROJ) INCLUDE_PCI_BUS
	vxprj build $(AMP0_PROJ)
	cp $(AMP0_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.0


amp1:
	vxprj create -force $(BSP) $(TOOL) $(AMP1_PROJ)
	vxprj bundle add $(AMP1_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP1_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP1_PROJ) INCLUDE_AMP_CPU_01
	vxprj component remove $(AMP1_PROJ) INCLUDE_WDB_SYS
	vxprj build $(AMP1_PROJ)
	cp $(AMP1_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.1


amp2:
	vxprj create -force $(BSP) $(TOOL) $(AMP2_PROJ)
	vxprj bundle add $(AMP2_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP2_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP2_PROJ) INCLUDE_AMP_CPU_02
	vxprj component remove $(AMP2_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP2_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP2_PROJ)
	cp $(AMP2_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.2


amp3:
	vxprj create -force $(BSP) $(TOOL) $(AMP3_PROJ)
	vxprj bundle add $(AMP3_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP3_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP3_PROJ) INCLUDE_AMP_CPU_03
	vxprj component remove $(AMP3_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP3_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP3_PROJ)
	cp $(AMP3_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.3


amp4:
	vxprj create -force $(BSP) $(TOOL) $(AMP4_PROJ)
	vxprj bundle add $(AMP4_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP4_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP4_PROJ) INCLUDE_AMP_CPU_04
	vxprj component remove $(AMP4_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP4_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP4_PROJ)
	cp $(AMP4_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.4


amp5:
	vxprj create -force $(BSP) $(TOOL) $(AMP5_PROJ)
	vxprj bundle add $(AMP5_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP5_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP5_PROJ) INCLUDE_AMP_CPU_05
	vxprj component remove $(AMP5_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP5_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP5_PROJ)
	cp $(AMP5_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.5


amp6:
	vxprj create -force $(BSP) $(TOOL) $(AMP6_PROJ)
	vxprj bundle add $(AMP6_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP6_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP6_PROJ) INCLUDE_AMP_CPU_06
	vxprj component remove $(AMP6_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP6_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP6_PROJ)
	cp $(AMP6_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.6


amp7:
	vxprj create -force $(BSP) $(TOOL) $(AMP7_PROJ)
	vxprj bundle add $(AMP7_PROJ) BUNDLE_STANDALONE_SHELL
	vxprj bundle add $(AMP7_PROJ) BUNDLE_AMP_SEC
	vxprj component add $(AMP7_PROJ) INCLUDE_AMP_CPU_07
	vxprj component remove $(AMP7_PROJ) INCLUDE_WDB_SYS
	vxprj component remove $(AMP7_PROJ) INCLUDE_NETWORK
	vxprj build $(AMP7_PROJ)
	cp $(AMP7_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.7

The important pattern is consistent across the projects:

CPU number
   ↓
AMP CPU component
   ↓
Per-core VxWorks project
   ↓
Build
   ↓
vxWorks.N

One important correction compared with many hand-maintained examples is to make sure vxprj commands reference the correct project file. For example, the CPU0 PCI component should be applied to $(AMP0_PROJ), not to an unrelated BSP project variable.

The exact bundle and component names are BSP- and VxWorks-version-dependent, so they should be checked against the target platform’s supported configuration.

🔐 Primary and Secondary AMP Images

#

The CPU0 image is generally different from the images running on secondary CPUs.

The primary image uses the AMP primary bundle:

BUNDLE_AMP_PRI

while secondary images use:

BUNDLE_AMP_SEC

A conceptual configuration is:

CPU0
 ├── VxWorks primary image
 ├── Primary AMP initialization
 ├── System management
 └── Image loading

CPU1
 └── VxWorks secondary image

CPU2
 └── VxWorks secondary image

...

CPU7
 └── VxWorks secondary image

This distinction matters because CPU0 participates in the initial AMP startup process, while the other processors are brought online by the primary CPU.

🚀 Loading Secondary Images with wrload

#

After CPU0 has booted, the wrload utility can be used from the VxWorks shell to load images for the remaining CPUs.

A simplified command looks like:

wrload("-file host:/vxWorks.1 -cpu 1");

A more complete example can include a bootline symbol:

wrload("-file host:/vxWorks.1 -cpu 1 -tsym \"*sysBootLine=dtsec(1,1) host:/vxWorks.1 h=192.168.1.100 e=192.168.1.51\"");

Additional CPUs can then be loaded in sequence:

wrload("-file host:/vxWorks.2 -cpu 2 -tsym \"*sysBootLine=dtsec(2,2)\"");

Add Delays Between Loads

#

When loading multiple secondary images, it is useful to insert a delay between wrload operations:

wrload("-file host:/vxWorks.1 -cpu 1 -tsym \"*sysBootLine=dtsec(1,1) host:/vxWorks.1 h=192.168.1.100 e=192.168.1.51\"");
taskDelay(60);

wrload("-file host:/vxWorks.2 -cpu 2 -tsym \"*sysBootLine=dtsec(2,2)\"");
taskDelay(60);

The delay gives the target CPU time to initialize before the next load operation begins.

The exact delay value should be determined experimentally for the target hardware and boot configuration rather than treated as a universal constant.

Bootline Configuration

#

The secondary image also needs a valid bootline when the application or BSP expects one.

The -tsym option can be used to provide the sysBootLine symbol to the loaded image.

A malformed or incomplete bootline can cause an otherwise correctly built secondary image to fail during startup.

For debugging AMP boot failures, verify:

• Image path
• Target CPU number
• Memory placement
• Bootline contents
• Network-device configuration
• Symbol availability
• Secondary CPU startup state

🧵 Accessing Secondary VxWorks Shells with tip

#

Physical serial connections are not always practical on a multi-core platform.

A system with eight VxWorks instances would otherwise require a large collection of serial connections just to access each shell.

MIPC serial devices provide another option.

Depending on the BSP, the system may expose devices such as:

/ttyMsd0
/ttyMsd1
/ttyMsd2
...

The VxWorks devs command can be used to inspect available devices:

-> devs

drv name

  0 /null
  1 /tyCo/0
  1 /ttyMsd0
  1 /ttyMsd1
  ...

The tip utility can then connect to secondary shells.

For example:

tip("dev=/ttyMsd0", "dev=/ttyMsd1")

This provides a convenient way to maintain multiple target sessions from CPU0.

tip Session Controls

#

A typical interactive session provides commands such as:

~.   Exit
~l   List sessions
~s   Switch session
~?   Help

The ability to switch sessions is especially useful when testing multiple AMP instances because the engineer can move from one CPU to another without changing physical connections.

🌐 Networking in an AMP System

#

Networking requires additional planning in an AMP environment.

Every VxWorks instance is logically independent, but the underlying processor and board may share physical Ethernet hardware.

This means the BSP and AMP configuration must account for:

• Ethernet-controller ownership
• MAC or interface assignment
• Memory regions
• Interrupt allocation
• Device access
• IP configuration
• Inter-CPU communication

In some systems, secondary CPUs may not need direct network access at all.

That is why an AMP image may deliberately remove general networking components:

CPU0 → Networking enabled
CPU1 → Networking enabled
CPU2 → Networking disabled
CPU3 → Networking disabled
...

Removing unused services can simplify the secondary image and reduce resource consumption.

However, this should be done only when the target application does not require networking on that CPU.

🔗 MIPC for Inter-CPU Communication

#

MIPC provides a communication mechanism for independent VxWorks instances running on the same multi-core platform.

It can be used to create logical communication paths between isolated operating-system instances.

A conceptual architecture is:

+----------------+       MIPC       +----------------+
|  VxWorks CPU1  | <--------------> |  VxWorks CPU2  |
|                |                  |                |
|   Application  |                  |   Application  |
+----------------+                  +----------------+

This allows individual applications to remain independent while exchanging selected data and control messages.

For legacy software migration, this can be particularly useful because applications do not necessarily have to be merged into one codebase simply because the underlying hardware has been consolidated onto a multi-core processor.

🧠 AMP Memory Planning

#

Memory partitioning is one of the most important parts of AMP configuration.

A basic memory model might look like:

Physical RAM
+----------------------------------------------------+
| CPU0 Private Region                                |
| VxWorks + applications                             |
+----------------------------------------------------+
| CPU1 Private Region                                |
| VxWorks + applications                             |
+----------------------------------------------------+
| CPU2 Private Region                                |
| VxWorks + applications                             |
+----------------------------------------------------+
| ...                                                |
+----------------------------------------------------+
| CPU7 Private Region                                |
| VxWorks + applications                             |
+----------------------------------------------------+
| Shared Memory / MIPC / IPC                         |
+----------------------------------------------------+

CPU0 often receives a larger allocation because it may run networking, management tasks, or additional system services.

Each secondary CPU needs enough memory for:

• Kernel image
• Kernel data
• Application image
• Task stacks
• Heap
• Device drivers
• Communication buffers

At the same time, the shared region must be large enough to support the intended IPC workload.

Avoiding Overlap

#

Every private and shared region must be placed carefully so that no two images accidentally overwrite each other.

A useful planning table is:

The exact physical addresses and sizes are BSP-specific and must be derived from the target board’s memory map.

🧪 Debugging AMP Startup Problems

#

AMP failures can be more difficult to diagnose than single-CPU boot failures because several independent systems must be correct at the same time.

A useful troubleshooting sequence is:

CPU0 Boots, Secondary CPU Does Not

#

Check:

• wrload command syntax
• Target CPU number
• Image location
• Memory placement
• AMP secondary configuration
• sysBootLine
• Secondary CPU reset/startup state

Secondary CPU Boots but Network Fails

#

Check:

• Whether networking components were removed
• Ethernet-device ownership
• Interface configuration
• Device-tree or BSP configuration
• MAC assignment
• Shared-resource conflicts

Secondary Shell Is Inaccessible

#

Check:

• Whether the MIPC serial device exists
• devs output
• MIPC/MSD configuration
• Correct /ttyMsdX mapping
• CPU startup status
• tip syntax

System Becomes Unstable During Multi-CPU Startup

#

Check:

• Memory-region overlap
• Shared-memory configuration
• Inter-CPU synchronization
• wrload timing
• Interrupt routing
• Device ownership
• CPU-specific BSP configuration

AMP debugging becomes much easier when each CPU can be observed independently.

📐 Automating an Eight-CPU Workflow

#

The main advantage of the Makefile approach is repeatability.

Without automation, an engineer may need to create and configure eight separate projects manually:

CPU0 → Create → Configure → Build
CPU1 → Create → Configure → Build
CPU2 → Create → Configure → Build
...
CPU7 → Create → Configure → Build

The automated approach reduces this to:

make amp

The build system then performs:

Create all projects
        ↓
Apply AMP configuration
        ↓
Apply CPU-specific components
        ↓
Remove incompatible features
        ↓
Build all images
        ↓
Copy vxWorks.0 ... vxWorks.7

This provides a much more predictable development workflow.

It also makes configuration differences explicit in source control.

✅ Conclusion

#

VxWorks 6.9 AMP provides a practical way to divide a multi-core embedded platform into independent operating-system environments.

Using an eight-core PowerPC platform such as the P4080, the architecture can assign each CPU its own VxWorks image and private memory while reserving shared resources for inter-processor communication.

The main building blocks are:

• Per-core VxWorks projects
• AMP primary and secondary configurations
• Private and shared memory regions
• Automated command-line builds
• wrload for secondary-CPU image loading
• MIPC for inter-CPU communication
• MIPC serial devices for remote shell access
• tip for managing multiple target sessions

The biggest practical lesson is that AMP configuration scales poorly when performed manually. Once a system contains four, eight, or more CPUs, automated project generation and repeatable builds become essential.

A Makefile-driven workflow turns each CPU into a reproducible software artifact, while wrload and MIPC provide the runtime mechanisms needed to start and manage the individual VxWorks instances.

For systems that need workload isolation, mixed operating systems, legacy application consolidation, or explicit per-core ownership, AMP provides an architecture that can be considerably more flexible than forcing every workload into a single SMP operating-system instance.