Table of Contents

I’ve been working on a data center migration from regular switches to a Cisco ACI fabric in the last couple of months. I can’t say that’s enough to be defined as an ACI expert but I’ll share here what I learned from the experience.

The project started with a Network Centric approach for an one-to-one migration from the previous network. After the migration new VRFs are being created in App Centric mode with contracts.

What is Cisco ACI?

I will not discuss the details of how ACI works here. This is only a general overview with some simplifications to keep the discussion flowing. Grab a copy of the awesome Deploying ACI: The complete guide if you’re interested to learn more.

What is Cisco ACI? Asks the network engineer

From a networker point of view Cisco ACI a big L2/L3 switch, call it fabric if you prefer. ipSpace provides a webinar about CLOS architecture that answers most questions.

That means moving from this physical topology



to this

The fabric supports all the main routing protocols (EIGRP, OSPF, BGP) and advanced PBR capabilities, named Service Graph.

The well known routed interfaces and SVIs are supported for peering with external L3 devices.

The ACI fabric does not run STP, it acts as an hub forwarding BPDUs. BPDU guard and BPDU filtering is supported.

MisCabling Protocol (MCP) is the loop protection mechanism of ACI, it’s quite similar to STP by the way.

Other protocols and functions not supported:

  • WCCP (use PBR instead)
  • NAT
  • STP (use MCP)
  • DHCP server (only relay works)

The experience I had with IPv6 on ACI was very positive. Every network function enabled was tested with native IPv4, dual stack and native IPv6 networks and all of them worked without any noticeable difference.

What’s new then? EPGs. This is a different approach to design and segment the network. I advise to fully understand how EPGs are created and how hosts are assigned to EPGs, this is the most basic and and the same time most powerful concept of ACI, the base of advanced features like contracts.

What is Cisco ACI? Asks the systems engineer

For the system engineer ACI means a simpler integration with the data center network. For the current project I implemented VMware and Microsoft SCVMM integration.

VMware integration was of course easier, mainly because there’s plenty of documentation available. SCVMM required more testing and thinking, documents are not very clear and I needed the support of a Microsoft system engineer to troubleshoot some issues. (Hint: set a subnet for the EPG - even if unicast routing is disabled for the BD - to send the correct subnet information to SCVMM).

The EPGs of ACI will be seen as port group (or equivalent) in the Virtual Machine Manager. This makes managing the VMs network assignment much easier and less error-prone.

When integrated with a VMM (Virtual Machine Manager) the ACI controller has full visibility of the VMs running on the hypervisors, their IP and MAC addresses, power status and more.

Read the Cisco ACI Virtualization guide for further information.

What is Cisco ACI? Asks the security engineer

Cisco ACI is the best friend of the security engineer. The Zero Trust model is the holy grail of security.

Endpoints in different EPGs can only talk if explicitly permitted.

The usual old-style security design requires to place endpoints in different subnets and route traffic between them through a stateful firewall to enforce security policies.

With ACI the EPGs can be part of the same broadcast domain - call it VLANs if you wish - but that’s not totally true. That means it’s easy to filter traffic between endpoints in the same subnet.

Note: contracts are quite L4 and stateless, they can filter based on TCP flags. Source and destinations are only EPGs, not subnets. They are more very similar to an ACLs on a switch.

This blog series and TFD video explain how segmentation works in ACI.

What is Cisco ACI? Asks the NOC engineer

NOC engineers has some specific needs in terms of features. They need a quick way to see the overall status of the network, faults, alarms, events and fix issues.

The APIC GUI helps a lot and it is quite intuitive - just click on the alarm to access a page with additional details. Every fault has a unique code, I found all of them in the Cisco ACI System Messages Reference Guide.

Operations like firmware updates and replacement of a faulty switch are intuitive and have no impact on the traffic when all the hosts are dual-homed, only a temporary downtime of one uplink.

What is Cisco ACI? Asks DevOps and automation engineer

Cisco API inspector and Visore are the best tools to start a programmatic approach to ACI configuration.

Postman can be used for quick creation of repetitive objects - the Collection Runner is a great feature indeed.

In the initial migration phase for every existing VLAN was translated as one BD and one EPG in ACI. Starting form a CSV file it was quite easy to find the correct API url and create all the objects on the APIC.

Plenty of code examples available in the ACI section of DevNet. and on Cisco Data Center GitHub repository.

The next step of my project will include an analysis of the repetitive tasks and the actual use of automation for Operations and Delivery of new services.

Sparse additional information

Naming

Naming objects in ACI is very important. Mainly because the name of the object is the primary key so it’s not possible to rename objects after the creation.

In the case of EGPs for example the same name is pushed to VMware as port-group. An easy to understand name is critical to avoid mistakes.

Sandbox

Cisco provides a free ACI sandbox available online. This is useful get confidence with the APIC GUI and test APIs. More labs available on dCloud.

Wrap up

My overall experience with ACI was very positive so far. I didn’t had a chance to implement multi site yet but I’m confident this will be the evolution and main reason of many ACI projects in the near future.

I think the full potential of ACI will be used when customers and consultants will have a better understanding of the product and its capabilities. The good part is it’s possible to start small using ACI as a L3 switch and integrate more functions later. The value of the product is immediate and increases over time when more automation is introduced.

Compared to previous deploys I did in the past with standard switches, ACI allowed me to spend less time on the console and more time talking with the customers about their business needs and how the network can help and provide value.

However the nerd inside me really appreciates the possibility of using commands in CLI to configure and display the output of the commands. ACI simplifies many operations but does not remove visibility when you need to see what really happens behind the scenes. The configuration guide shows for every feature how to do it via GUI, API and CLI.

I expect ACI will be part or many networks very soon and I welcome this shifting of the network engineer role with more involvement in the business processes. It’s time to evolve roles and networks, better be ready.