Introduction

Backbone network flow monitoring system is mainly aimed at monitoring network of provincial network operators, with the flow generally of more than 100G. Architecture of network monitoring system is similar to the traditional IDS, that is, by mirror image of the flow to the monitoring system through an optical splitter or special flow backflow devices. Due to great flow, it is necessary to configure the specialized primary bypass device and secondary flow load balancing device for bypass flow, and then for processing by connecting to each processing equipment. High requirements are put forward for computing power and network IO capabilities of a single processor. Therefore, Sugon has specially developed Flowfirm to deal with the problems of access and bypass, and hardware devices such as Netfirm and videospeed to improve capacity of handling network flow of the processor. With these methods adopted, new changes are also brought to construction of traditional Internet monitoring system.

Scheme framework

Sugon backbone network flow monitoring system is divided into three levels: the front level bypass equipment makes the mirror image processed, and they are distributed to each server with balance according to certain principles. The front-end processing platform is configured with Netfirm front end processing probe to deal with flow. Considering the future expansion of business, flow that cannot be dealt with by the front-end probe will be transferred to the backend; front second level processing platform is designed with a network processing machine with high speed co-processor card for further mining and analysis. Shunt equipment has the function of flow replication at the same time, which is able to copy the specific flow and transfer it to monitoring network in other parts.

Scheme features

Flowfirm-S flow is connected to bypass flow devices

There are a variety of flows in backbone network, such as effective data packets, invalid packets, etc.. If it is directly sent to server processing platform for processing, it will not only increase pressure of server processing, but also will increase the pressure of the entire link, which is also a severe test on packet processing and classification capability of software business system. Under such a background, Flowfirm-S can carry out converge, preliminary analysis and processing of large flow of the backbone network, while the screening concerned by the business will be handed over to the subsequent treatment system.

Flowfirm-S is developed based on ATCA specification of international open telecommunications equipment standard. In view of special telecommunication level comprehensive flow equipment required by communication operators and safety unit of governments at all levels, it is mainly used in all levels of POS/SDH and Ethernet line for flow collection, filtering, converging and bypass flow and planning function of other complex flow.

Main functions of Flowfirm–S are:

1) Flow switched in：Support a variety of interface cards, support single-mode of OC3, OC12, OC48, OC192, GE and XE, as well as multimode optical fiber link, be accessed with 40G POS、10G POS、10GE、2.5G POS、622M POS、155M POS、GE/FE, etc., and support mixed access, high density connection. The single plate access density reaches 1 route 40 G or 4 routes 10 G or 16 routes 2.5 G. Receipt and acceptance of single fiber is also supported.

2) Exchange protocol support：mainly used for POS interface; support Ethernet based on SDH/SONET, PPP and CHDLC encapsulated packet; support IPV4 / IPV6 format message s of MPLS encapsulation and VLAN encapsulation and support stripping; by parsing HDLC, PPP, MPLS encapsulation, extract IP packets, and make subsequent processing. It is compatible with a variety of underlying encapsulation parameters.

3) ACL (packet matching classification)：support flexible mask rules based on quintuple group; support ACL function (DPI) with mask data matching extension based on the loaded specified offset, support both IPV4 and IPV6 capabilities, support six flexible quintuple group rules of 2 million (IPV4), and 1 million (IPV6) defined by users, support mask rules 600000 (IPV4) and 200000 (IPV6), support 32 kinds of priority among different rule types, support binding between standard ACL and extended ACL, classification rules are managed according to rule set, and users can create and set up multiple rule sets which can be bound to multiple interfaces.

4) Flow filtration, convergence and load balance distribution：messages hitting ACL rules are supported to be abandoned , pass-through, shunt forwarding, flow replication, message truncation and sampling distribution; message distribution supports a variety of load sharing algorithms (polling, Hash, quintuple group, etc.); nodes within shunt group can be based on the hardware port, and can also be based on the back-end server IP address; dynamically monitor back-end server statue or port state with dynamic update of shunt group, and ensure homologous homoclinic as far as possible; support 32 shunt groups, in which 256 nodes are supported at most; output source MAC support information carrying capabilities of message.

5) Flow replication：support flow replication at the same rules: namely, specify replication method for the flow that conforms to a rule; under the premise that does not affect flow distribution strategy , the flow can be copied to another or a group of GE /10GE output.

6) Re-encapsulation of data package：re-pack the processed IP package that conforms to final standard of Ethernet protocol, and carry input interface and other information; support to truncature and extract useful head of output package.

7) Webmaster function：support SNMP, Telnet, ssh and other webmaster functions.

Figure Flowfirm-S Shunt device

Flowfirm–S has the following features：

1) Strong adaptability

Provide cost-effective single board or double plate schemes for monitoring of low link density; provide large capacity solution based on back veneer exchange for application of high link density.

2) Telecommunication level standard

Based on the highest standard ATCA of telecommunication level open platform, it provides products with high reliability based on the standard, has excellent chip/blade heterogeneous ability allows to configure network processing blades, universal calculation blade and various professional business blade (such as DSP voice communication blades, etc.), protecting users’ investment.

3) Transmit of overall hardware

Hardware processing with multi-engine assembly line ensures wire-speed forwarding of data. Many engines are designed for each part of packet processing assembly line.

4) Diversity of application

It can meet the needs of filter matching and application processing of high complexity, and cater to constantly complicated emerging network applications.

5) High cost performance

As shunt equipment for optimized design, it can make function customization according to actual needs of users and network environment.

6) Data processing function

It includes a variety of interface flow access, protocol conversion, data packet matching classification, filtering flow, flow convergence and distribution and load balancing, flow replication, packet encapsulation, etc.

7) Support cascade extension of devices

Break through chassis slot capacity of single equipment system, and cascade can be extended based on demands, providing great space to support the future network development.

Figure Flowfirm – S device cascade

Improve single processing capability by Netfirm

Netfirm is a high performance intelligent network accelerator card specially designed for network flow monitoring project. Netfirm works based on FPGA with built-in 4 GB memory for data processing; high-speed access and uninstall of CPU load and other functions can be realized by using Netfirm; it is a powerful tool to solve many problems of high network flow currently under the condition of increasing network flow.

Figure Netfirm smart acceleration card

Advantages of Netfirm for network monitoring include:

1）0% CPU consumption, and 100% capture of data package

Common network card requires a lot of CPU resources in the process of receiving data, and with the increase of network flow, the consumption will become very considerable, so that network data cannot be completely gained, which may have impact on monitoring effect of business systems. For this reason, single processing power used by traditional Internet surveillance can only be about 500 MBPS. Netfirm intelligence acceleration card will make the whole data receiving process automatically realized, so as to ensure complete data acquisition while without consuming any CPU resources; there is no any packet loss even with 10 Gbps flow processed.

2）Computing power of each CPU fully played by bypass flow technology

With transfer of CPU technology from main frequency increase to multi-core technology, how to make good use of so many CPU cores has become an important problem for Internet monitoring system. According to the characteristics of network monitoring system, Netfirm will make bypass flow for received shunt flow according to IP information of source destination, so that processing load makes balance among multiple CPU cores, which makes full use of multi-core CPU processing power.

3）Flexible and rich filter criteria will further decrease CPU load

Another idea to increase the capacity of single processing is to filter the flows that are simple and don't need to be processed, so that host resources can be concentrated to all effective flow processing. Based on this idea, Netfirm can be configure with 200K IP filtering rules, and many actions like discarding, forwarding and marking for different links, so as to further improve capacity of single processing.

4）Support seamless fusion of many applications to improve resources utilization rate, convenient for extension in future

At the early stage of Internet monitoring system construction, basically, it was to use a set of hardware construction method with application. However, as the system gradually matures, users need more features and applications, and a series of problems such as room space and hardware procurement costs will occur for reuse of the method. The best way to realize the application is to run a variety of different businesses on a set of hardware, and they won’t influence each other, which cannot be realized in traditional network monitoring system. But from the beginning of the design, Netfirm has considered the possibility of expansion of network monitoring system to support multiple different applications to obtain data from Netfirm at the same time, and guarantee the isolation, thereby significantly reducing the costs in expansion of network monitoring system.

5）Universal interface and special interface based API, convenient for transplant of application

Many current applications of Internet surveillance system are based on ordinary network card in the process of research and development. Therefore, Netfirm supports the same modes with ordinary cards, and can provide the same access interface. Special interface is only needed when Netfirm additional functions are needed, which thereby greatly reduces the transplant cycle of Internet monitoring applications.

Second level data processing based on Videospeed

Sugon Videospeed co-processor card is a co-processor card based on many-core technology developed for monitoring that cannot be processed in traditional network monitoring system, such as audio and video flow monitor and other non-text data. The card can provide high-speed parallel processing, and can support decoding processing of multiple audio and video stream, thus to process the audio and video flow that cannot be processed by ordinary Internet surveillance system in the past processing. It can also be used as a general-purpose computing acceleration platform, used for depth data processing for other specific flows.

Figure Videospeed accelerator card

Videospeed co-processing card has the following features:

1) Based on many-core technology： Tilera many-core processor is adopted, which can greatly simplify the system framework, costs, power consumption and PCB area; it can accurately allocate resource to realize the planned function based on single core processing capacity, reaching optimizing performance and saving power consumption.

2) Strong processing capacity：Complete a certain decoding of flow audio/video mixed flow, which can support more than 400MB flow.

3) Support many mainstream video formats：mainstream video format H264，H263，Mpeg1/2，Mpeg4，vp6，flv，asf，avi, etc.

4) Support vector calculation：Acceleration of a certain vector calculation and other function calculation

High performance network processing all-in-one machine

Sugon high performance network processing all-in-one machine can integrate flow processing probe and secondary processing platform in backbone network monitoring solution into an all-in-one machine; a single machine can provide computing capacity of more than 10 pieces of dual roads and many cores processor, designed with 6 piece Netfirm, which can support 24 root GE, or 6 10 GE flow monitoring. 4 videospeed co-processor cards are configured, which can not only carry out secondary processing of the flow, but also can undertake 10GE with all configuration of Netfirm of flow processing, providing a processing capacity of 100G flow of a single all-in-one machine with very high cost performance.