Switchbox Routing

Switchbox routing is implemented in the DP-FPGA (See The DP-FPGA Architecture), the Raw machine (See The Raw Machine), and Pleiades (See The Pleiades System). As already discussed in See The Raw Machine switchbox routing is very flexible, but opens an infinite design space for application mapping. This is already known form the C.mmp project, which was undertook by the Carnegie-Mellon University in 1971 See [SBN82].

Multi-Length Wiring

Multi-length wiring is implemented by Matrix (See The MATRIX Architecture) and MorphoSys (See MorphoSys Reconfigurable Cell Array). While it is flexible, it opens a large design space, which results in a lack of application development support. A further disadvantage is a certain overhead of chip area needed by the long range connections.

Routing Channel

The RaPID architecture (See The RaPiD Architecture) is a linear array of datapath units, which uses segmentable busses as a kind of routing resource. The simplicity of this architecture supports automatic application mapping. The limited design space supports automatic application mapping, but the limited capacity of the routing channel restricts the design space.

Nearest Neighbor Connections Only

The CHESS array provides nearest neighbor routing resources only. As known from the Algotronix CAL1024, the availability of nearest neighbor connections only may lead to routing congestions (see introduction of this chapter). Therefore long distance routing may be expensive in systems with nearest neighbor connections only. But this rule does not necessarily hold for coarse grain platforms. In coarse grain reconfigurable systems it is feasible to have rich routing support also by cells already assigned to functions. In such systems the restricted design space eases application mapping.

Nearest Neighbor Connections with Global Backup-Bus

Nearest neighbor connections with a global backup-bus is implemented in the KressArray. It has turned out that a simple local routing scheme without much long range routing is sufficiently powerful for high speed communication in coarse-grained reconfigurable architectures. Furthermore it eases application mapping efficiently See [HHH00]. Routing congestions, like known from architectures with nearest neighbor connections only, are avoided by the utilization of a global backup-bus.

Therefore in this thesis the KressArray-3 with no embedded memory will be used as a basis to develop a reconfigurable device / memory infrastructure to alleviate the memory communication problem. The KressArray-3 with the Xplorer Tool See [HHH00] provides very high flexibility in designing array communication architectures, which also supports a rich choice of possible array memory interface architectures. It has been a goal of this thesis, to exploit this flexibility for very powerful memory interfacing with special address generator support. This has been a challenge and a chance to find a better solution than known before. To cope with the memory bandwidth problem associated with KressArray use there are two degrees of freedom: to influence the KressArray design, and to develop a memory interface architecture for it.

Synchronous DRAM

The first DRAMs which operated synchronously to the system clock were Synchronous DRAMs (SDRAM, See [Pri96]). SDRAMs support burst read accesses with a programmable burst length of 1, 2, 4, 8 words or full page. Since the programming of the burst length is costly, it is done only once. SDRAMs are organized in two or four banks, which allows to hide partially precharge and permits interleaving between the banks (see See [Pri96] for detailed information).

Multibank DRAM

Multibank DRAM (MDRAM, see also See Multibank DRAM Technology) has been developed from SDRAM by Mosys See [MoS96] and was fabricated under licence by Siemens Semiconductor See [Sie96] See [Sie97]. MDRAM integrates 32 or 36 banks, which allows to hide activation and precharge of banks completely and permits interleaving between all banks. Data is transmitted on a 16 bit bus at both edges of the clock signal, which doubles the memory interface bandwidth.

A very interesting feature is the variable length burst, i.e. the burst length is not fixed and can be different for each read and write access. Therefore MDRAM is not programmed by a specific burst length like SDRAM but has a command bus, where a stop signal is sent. While this feature makes the MDRAM more flexible, the memory controlling is quite complicated.

Double Data Rate Synchronous DRAM

The Double Data Rate Synchronous DRAM (DDR-SDRAM, See [All98] See [Jed99]) has also been developed from SDRAM to compete with Rambus technology. The double data rate technology transmits data on both edges of the clock signal. DDR-SDRAM technology is cheaper than Rambus DRAM and expected to be of similar performance. DDR-SDRAM integrates four internal banks to improve hiding of activation and precharging compared to SDRAMs. It supports 2, 4, or 8 word read and write bursts. The burst length is programmed only once at power-up.

Rambus DRAM

Rambus DRAM (RDRAM, See [Ram99a] See [War99]) was developed from the traditional DRAM, but the architecture has been streamed and optimized to yield new performance. Data is read in packets of 16 bytes at a very high clock speed. The RDRAM modules are only 16 bit wide, but they transmit data at both edges of a 400MHz clock. Therefore the RDRAM chips have to be placed very close to the CPU to reduce radio noise.

Conclusions

See Summary of the technical details of the different DRAM types. summarizes the technical details of the presented DRAM types. All of the above DRAM technologies provide enough capacity to implement a sufficient large data memory. Because of SDRAM is the precursor of MDRAM as well as DDR-SDRAM, DDR-SDRAM and MDRAM have been further optimized and provide generally a better bandwidth. The main reason for this is the increased number of internal banks and the double data rate interface. Since the burst length of DDR-SDRAM is pre-programmed, it is less flexible than MDRAM. While the pre-programmable burst length is good for the hierarchic memory of microprocessors it limits the flexibility of reconfigurable accelerators.

The Rambus technology hides the memory device from the processor in a way that multiple memory requests may be performed at a time and carried out later by the Rambus controller. This proceeding is unsuitable for generic data sequencing. Here a closer coupling between the memory device and the data sequencer is desired to perform optimized accesses.

Reconfigurable System With External Memory

Reconfigurable systems with external memory differ in the type of utilized memory and in the number of memory blocks. A simple solution for data memory is SRAM. It is easy to handle (e.g. no refresh, precharge etc.) and has short access times. The disadvantages of SRAM is the price and the device size.

To implement large data memories, necessary for many applications, DRAM suits better than SRAM. It is cheap and available in smaller devices. The disadvantage is, that DRAM needs a sophisticated memory controller to generate refresh cycles and to perform the more costly accesses (i.e. perform bank activation and precharge and so on). Due to the well developed DRAM technology some reconfigurable systems already benefit from access optimization techniques: e.g. PRISM-II (See The PRISM-II System) performs accesses in burst mode, or Riley-2 (See Riley-2) uses Burst Enhanced Data Out (BEDO) DRAM See [Mic95a] See [Mic95b].

A popular hardware level access optimization in reconfigurable systems is concurrent access to parallel memory. Several independent memory banks are attached to the reconfigurable datapath expecting that data can be clever distributed and accessed concurrently. Unfortunately this leads very often to inconsistencies or a large routing overhead. This method can be applied to SRAM-based systems as well as to DRAM-based. As a result reconfigurable systems with external data memories can be grouped in the following classes:

Reconfigurable Architecture With Virtual Internal Memory

Reconfigurable architectures with single or a few irregular internal data memories are conventional FPGA architectures with reconfigurable resources allocated as memory. Usually the memory of multiple look-up tables is united to larger memory blocks. Many commercial FPGA architectures integrate this feature. Some examples are the Xilinx XC4000, Spartan and Virtex families See [Xil99]. Because of the method how this kind of internal data memory is implemented, it is very area inefficient. This approach is only used for small memories, e.g. to store intermediate results.

Reconfigurable Architecture With Embedded Memory

Also many reconfigurable architectures integrate real SRAM blocks into the same device. Usually such memories are spread regularly over the architecture. Since the memories are integrated in between reconfigurable media, they are also rather small.

Most of the coarse-grained architectures presented in See Coarse-grained Configurable Architectures (RaPiD, MATRIX, Raw and CHESS) integrate this kind of internal memory, but also some fine-grained architectures. The remaining part of this section sketches two commercial FPGAs with embedded SRAMs.

The Altera FLEX10k CPLD Family

The Altera FLEX10k CPLD family is a fine grained architecture (see See [Alt98]). The CPLD has SRAM cells (embedded array block, EAB, see See Altera FLEX 10k CPLD family device block diagram with embedded array blocks (EAB).), which can be used as memory as well as function generators. Each EAB provides 512x8 bits.

The Actel ES Family

The Actel ES FPGA family presented in See [KBK99] is a fine-grained architecture with embedded SRAMs. Each B16x16 tile (See The Actel ES FPGA: (a) overview on a 4-tile device, and (b) B8x8 utilities with embedded SRAM.) has 2 blocks of 512x9 bit SRAM (See The Actel ES FPGA: (a) overview on a 4-tile device, and (b) B8x8 utilities with embedded SRAM.).

Reconfigurable Architecture Surrounded by Memory

This class of reconfigurable devices with internal memory uses a regular block of reconfigurable media, which is surrounded by memory. In this thesis a KressArray architecture surrounded by memory blocks is considered (see See The Data Sequencer Mapped to the KressArray). A fine-grained commercial FPGA following this approach is the Xilinx Virtex family:

The Xilinx Virtex FPGA Family

The Virtex FPGA See [Xil99] shown in See Xilinx Virtex series [Xil99]: (a) architecture overview, and (b) BlockRAM size of different devices. comprises two major configurable elements: input/output blocks (IOBs) and configurable logic blocks (CLBs), which can also be allocated as memory (see See Reconfigurable Architecture With Virtual Internal Memory). The VersaRing I/O interface provides routing resources around the periphery of the device. The architecture also includes clock DLLs (delay-locked loop) for clock distribution delay compensation and clock domain control.

On both sides of the CLB array, the Virtex architecture has dedicated Block RAMs of 4096 bits (BRAMs). BRAM memory blocks are organized in columns. All Virtex devices contain two such columns, one along each vertical edge. These columns extend the full height of the chip. Each BRAM memory block is four CLBs high, and consequently, a Virtex device 64 CLBs high (see XCV1000 in See Xilinx Virtex series [Xil99]: (a) architecture overview, and (b) BlockRAM size of different devices.b) contains 16 memory blocks per column, and has a total of 32 blocks.

Conclusions

The number and variety of useful memory products on the market is growing. Therefore there is a good chance to solve the more difficult memory bandwidth problem for powerful accelerators. As the semiconductor continuously increases the integration density, sooner or later memories will be placed as IP cores on the same chip as the reconfigurable platform.

Currently most existing reconfigurable accelerators are based on the external data memory concept. The limited number of off-chip pins and the reduced speed of off-chip communication limits the data throughput. This leads to a discrepancy between data throughput and memory bandwidth. Therefore such accelerators struggle with a similar bottleneck as known from von Neumann systems.

The main reason for these implementations is, that most commercial reconfigurable devices still integrate only a little internal memory. But with the increasing chip density internal data memory becomes feasible. The trend to IP cores and systems on chip (SoC) confirm the use of internal data memory. Internal data memory implementations are not limited by device pins and achieve higher access speed because of the on-chip communication. Using internal memory blocks, the number, organization, and integration style of internal memories influences the memory bandwidth and the efficiency of application mapping support. HDL support is often area inefficient and complicated to use, if internal memory blocks are utilized. But this depends on the mixture of memory and logic:

Many FPGAs support to allocate reconfigurable resources as internal data memory. While this method complicates application design and results in an inefficient placement and routing, it provides only very little memory resources. Therefore this approach is not suited to implement data memories of feasible size for reconfigurable computing.

The embedded memories can be seen as a coarse grain logic in memory approach, which brings the memories into the reconfigurable datapath. But the mixture of logic and memory makes the implementation of a global control difficult. Also the programming from outside and the union of multiple banks to form a larger memory is complicated. This demands a lot to an associated application mapping software and results in suboptimal mappings.

A reconfigurable architecture surrounded by data memory on the same chip avoids a mixture of logic and memory. Currently it is the most promising approach of internal data memory. Longer communication paths on chip are compensated by easy access from outside and the ability to form larger memories through the union of multiple banks. Application mapping is simplified, because the reconfigurable cells are not fractionated by the memory blocks.

Data Types

The SMA implementation distinguishes among four types of operands: (1) immediate operands, (2) scalar operands, (3) data structure operands, and (4) index operands. Immediate operands are data whose values are embedded in an instruction. An index operand is one of the current indices found on the index stack. The index operand is used only to read a current index value from the index stack and transfer its value to the CP. An index operand differs from a scalar operand primarily in that the index operand originates from the MAP while the scalar operand originates from the memory. The operand type may be specified in a subfield of an instruction's operand field or it may be implicitly associated with a particular instruction. Additionally, an indirect addressing mode is provided specifically for use in the calling of subroutines and in the accessing of data items from structures such as linked lists. As with operand type specification, indirect addressing may be specified in a variety of ways.

At some time it may be desirable to distinguish explicitly among several types of data structures. Instead of having a data structure operand, one my wish to have an array operand, a linked list operand, a binary tree operand, etc. For each operand type, some special accessing mechanisms would be provided to improve the speed with which an operand address is generated. Accessing mechanisms as implemented allow the instruction code to reference structured data simply by pointing to the mechanism, which then references the next data in the established pattern.

Scalar Data

Scalar data is treated in the manner of a vector rather than as a set of disassociated items. The specification for a scalar operand includes a specification of a MAP base register and a displacement into a scalar data area in memory. The MAP can have more than one scalar base register to aid in the accessing of local and global variables, such as during subroutine calls. Such a base register can be used as the argument pointer during a subroutine call.

Index Operations

The SMA's memory access processor has special mechanisms to track indices for data structure computations. These indices are used to generate the addresses for specific items of the data structure to be referenced. An index is specified by its current value, final value, step-size and indexing level. When the index is first established, the current value is equal to its initial value. At any time, several indices may be active; and the level, or nesting, of these indices is dictated by the time at which they were instantiated, or set up. In the SMA, the current value, final value, and step-size of an index are kept on a LIFO (last in first out) stack structure known as the index stack (IS). Each stack position is numbered sequentially, with the bottom of the stack numbered level 1. This convention provides a convenient way of referring to the current value of any active index because the bottom of stack entry corresponds to the outermost level of nesting, i.e. level 1. Stack continuation in memory can be provided for overflow.

When a setup index instruction is executed by the MAP, the initial value, final value, and step-size are pushed onto the stack. To change the current value of an index, an increment index instruction is used. This instruction must specify three items: the level of the index which is being incremented, and two initial addresses of blocks which are the targets of a branch outcome. If the current value of the index is less than the final value, control is transferred to the first block which is specified. If, on the other hand, the current value equals or exceeds the final value, control is transferred to the second specified block.

By checking the index value early, during each increment index instruction, and by having the branching information available, the next instruction can start being accessed while the CP is still performing the final computations of a loop. Furthermore, no guess is made about which direction an index-based branch will take, thus no time is wasted in fetching potentially unnecessary blocks of instructions from the memory.

When the current value of the index has reached its final value, that index should be at the top-of-stack and it is removed (popped) from the stack. Two other methods of removing indices from the stack are (1) the "remove index" instruction which removes the highest level current index from the top of the stack and (2) a "clear all indices" instruction which removes all indices from the stack.

To save loading of index instruction parameters from memory, the MAP is loaded with a set of templates for these values at the start of program execution. A template is a specification of the values needed to initialize an index on the index stack. Templates are loaded into an index template table. When an index is set up in the IS, the IS is loaded directly from the index template table. For a particular program, the number of distinct templates could be fairly small. For example, analysis of a Gaussian elimination program shows that 995 dynamic index setups are required, representing 16 static index setups, but only 8 templates are needed. Each index activated with a particular initial specification can use the same entry from the index template table. Even if the number of templates exceeds the table size, judicious reloading limits overhead.

The JumpGenerator of the MoM-2

The data sequencer of the MoM-2 is based on a single JumpGenerator (see See The JumpGenerator of the Map-oriented Machine 2.) which computes the base location of the cache frame in the data memory. The x- and y-address is generated generically on the basis of a few parameters, which are explained in detail in See [Hir91]. Therefore it utilizes two identical 1-dimensional address generators which are synchronized by a special control logic.

Each 1-dimensional address generator holds three identical steppers (see See Map-oriented Machine 2 Stepper Architecture.). Each stepper is programed by an initial value which is modified by positive or negative number in each step. The Stepper is running in a loop and generates addresses until a maximum value is reached. While the Base- and Limit-Stepper operate on parameters given by the Task Manger, the Address Stepper obtains its initial and maximum values by Base- and Limit-Stepper.

The JumpGenerator may generate basic 2-dimensional access pattern without interaction of the next higher control instance. It has no fixed step-width and also the loop-boundaries can be set at random. For complex access pattern the control of the Task Manager is required. For instance an access pattern like the JPEG zig-zag scan requires frequent exchange of the parameters.

To describe the access pattern generation of the JumpGenerator in See [HHW90] the slider model has been proposed. This model visualizes the generic address generation process based on a small parameter set. The slider model will be explained in detail in See Address Generation with the Slider Method.

The Single Step Control Unit of the MoM-2

The Single Step Control Unit (SSCU) controls the updates of a scan cache of four by four data words on the basis of the base address provided by the JumpGenerator. The size of the scan cache may be reduced by skipping away complete rows or columns.

The updates of the Scan Cache between two steps may affect the complete cache or may be optimized with a shift register structure. The optimization takes place if subsequent positions differ only by one horizontal or vertical direction. In that case only the new row or column is updated, while those overlapping with previous scan cache positions are shifted to their proper location. Only at the beginning and the end of loops in the JumpGenerator, a full cache update is performed. By this method many memory accesses can be saved.

Internally the SSCU utilized two 2-bit counters to generate the memory addresses. These counters are controlled by a FSM based on the programming from the Task Sequencer. See The Single Step Control Unit of the Map-oriented Machine 2 for scan window generation. shows the basic architecture of the SSCU.

The Overall Architecture of the DSP

See The video signal processor (VSP) as a basis for the utilization of the address generation unit (AGU). shows the overall architecture of the DSP which includes the AGU proposed in See [KOD91]. An important architectural feature is the employment of independent AGUs and execution unit (EU).

The three AGUs are implemented to calculate the address for external image memory. The number of AGUs, three, is obtained by analyzing how many image memories are used in video signal processing. The constitution of the AGUs independent of the EU is effective, because the EU may concentrate on calculations for actual data, and therefore, the AGUs and the EU can operate in parallel.

Addressing Modes of the AGU

Seventeen addressing modes are prepared in the AGU (See VSP address generation unit (AGU) addressing modes. and See VSP address generation unit (AGU) sub modes of neighborhood search. show the addressing modes). They were defined by analyzing image processing algorithms. The relationship between the analyzed typical image processing algorithms aid their associated addressing modes. Seven initial parameters, e.g., the start address, the end address, the increment value, and the aspect ratio of the image plane, have to be specified before execution.

The 2-dimensional raster scan mode (See VSP address generation unit (AGU) addressing modes.(1)) is similar to tile TV raster scan method, and it is used in most image processing algorithms. In this 2-dimensional raster scan mode, image data are accessed from the left edge to the right edge along the scanning line. When scanning reaches the right edge, it returns back to the left edge of the line just below the previous one and begins to scan again.

The block scan mode (See VSP address generation unit (AGU) addressing modes.(2)) is suited for spatial filtering. In the block scan mode, the local area kernel itself slides like in a 2-dimensional raster scan while the image data are scanned within the local kernel.

The neighborhood search mode (See VSP address generation unit (AGU) addressing modes.(3)) is used in algorithms, like labeling, region segmentation, and border following. In the neighborhood search mode, the AGU successively generates the neighbor addresses of an address which is generated by other methods such as the indirect access mode. Eight variations to generate these neighbor addresses are prepared in the neighborhood search mode.

See VSP address generation unit (AGU) sub modes of neighborhood search. shows the eight variations of the neighborhood search mode. In See VSP address generation unit (AGU) sub modes of neighborhood search., the number in each grid means the accessing order. For example, in the NAC8 mode, the eight neighboring points are accessed according to the accessing order. This accessing method was employed by considering the adjacent definition in image processing. Furthermore, it is effective to use from NRS8U to NRS4L modes combined with the 2-dimensional raster scan mode in some particular image processing applications.

The 2-dimensional indirect access mode (See VSP address generation unit (AGU) addressing modes.(4)) is used when the AGU outputs an irregular address which is calculated by using the arithmetic unit in the EU. In this mode, the AGU adds the offset values to the results of the arithmetic units, and generates the address of the external image memory.

The 1-dimensional raster scan mode and the 1-dimensional indirect access mode (See VSP address generation unit (AGU) addressing modes.(5),(6)) are 1-dimensional versions of the 2-dimensional raster scan mode and the 2-dimensional indirect access mode, respectively.

In addition to these modes, the FFT8 mode and the affine transformation mode are prepared as special modes. In the FFT mode, not only the addresses of a 1-dimensional FFT but also those of a 2-dimensional 1FFT can be generated.

The Architecture of the AGU

See The VSP address generation unit (AGU) architecture. shows the constitution of the AGU. The AGU consists of 32 bit counters, 32 bit adders, barrel shifters, a bit reverse circuit and a neighborhood access decoder. It is structured as a three stage pipeline. Barrel shifters are used for changing the aspect ratio of a 2-dimensional image plane. This barrel shifter can perform both left and right 24 bit shifts, and therefore the aspect ratio of the image plane can be changed from a 1 bit x 16M bit narrow figure to a 16M bit x 1bit wide figure. A bit reverse circuit which can adjust the bit reverse width is used in the FFT addressing mode. The neighborhood access decoder generates the relative distance between a central point and neighborhood points. The neighborhood addresses are generated by adding the distance to the address of the central point which is assigned from the internal data bus according to the appointed neighborhood access mode described in See VSP address generation unit (AGU) sub modes of neighborhood search..

The selectors in See The VSP address generation unit (AGU) architecture. are controlled according to the appointed addressing mode. The maximum value, minimum value, or increment value, of the counter is set by initial parameters.

The Handle Position Generator

The Handle Position Generator (HPG) has been derived from the MoM-2 JumpGenerator with some improvements but it is still based on the same address generation method. The programming model of the memory is a two-dimensional map. Therefore the HPG consists of two identical parts, one for each dimension (See Handle position generator of the MoM-3 generic address generator.). The so-called 1-D Address Generators operate in parallel and synchronize through a trigger logic, which preserves the symmetry of the design. The programmer decides, which (if any) 1-D Address Generator operates as master, triggering the other 1-D Address Generator to perform a step. The trigger logic routes the trigger signals according to the programmer's specification. Furthermore, it evaluates all conditions, which require a knowledge of the state of the complete system (e.g. when a scan pattern terminates).

The one-dimensional positions produced by each of the 1-D Address Generators are checked against segment limits, before they are presented to the Memory Address Generator as valid handle positions. This serves as a kind of memory protection scheme, which is especially useful, if the handle positions are computed under control of the data manipulating devices, dependent on the data processing results. The segment check units make sure, that all handle positions of a GAG remain within a programmer-defined orthogonal bounding box in the two-dimensional memory map. By providing the maximum and minimum offsets that occur in the Memory Address Generator program, the segment checks can even evaluate whether the bounding-box would be left during the generation of memory addresses and invalidate such a handle position before it is passed on in the pipeline. The actual address computations are done by the 1-D Address Generators.

The Memory Address Generator

In contrast to the HPG the Memory Address Generator (MAG) is a great improvement in flexibility of the MoM-3 compared to the MoM-2. The MAG resembles a kind of RISC processor with a special purpose instruction set and on-chip instruction memory. The only task that has to be performed by the MAG, is to provide an arbitrary sequence of offsets to be added to the handle position to perform memory accesses in a local neighbourhood around the scan window handle position. The most efficient way to support arbitrary offset sequences, is to make the offsets directly programmable in a memory that is scanned linearly. At every new handle position, the memory is scanned again from the beginning. But since the length of the offset sequence varies from application to application, an escape mechanism has to be programmed into the offset memory, to signal the end of an offset sequence. Interpreting the offsets as read or write instructions and the escape code as branch to the beginning, the MAG can be programmed with a small and simple instruction set. Additional instruction codes have to be introduced to support pipelining in the data processing devices with the requirement of conditionally executed read or write operations at the beginning and at the end of loops, to fill the pipeline, and to flush results remaining in the pipeline.

The range of offsets may be -32 to +31 in both dimensions of the data memory. The memory to store the instructions (address instruction RAM, AIR) has 256 entries, so that a maximum of 253 references to the data memory can be made from a single handle position. At least three instructions are overhead, which have to be inserted to accept a new handle position from the Handle Position Generator, to signal the start of computations to the data processing devices, and to jump to the beginning at the end of an offset sequence.

After the instructions have been fetched and decoded, the offsets have to be added to the current handle position. A number of steps are performed with each of the resulting addresses, to transform them into physical memory addresses. These tasks are completely hardwired, because they have to be applied to each address. The first is an address modification to allow for cyclic addressing. For each dimension, a CycleMask word is used to keep selected bits of the address from changing, and a CyclePattern word provides default values for the masked bits. If the CycleMask is partitioned into a leading block of 16-k zeroes and a trailing block of k ones, for example, the resulting addresses automatically wrap around at the 2 k boundary, because the values of the higher (16 - k) bits are masked. Input to the following step are still two 16-bit address words, one for each dimension. To be able to access a conventional linear memory, the address parts have to be combined to a linear memory address. The address parts of the two dimensions may be combined to a real linear memory address in four ways: one row of two-dimensional memory (x dimension) may consume an address space of 10, 12, 14, or 16 bits. This allows to adjust the "size" of the data memory to the size of the processed data, to reduce wastage of address space. After the concatenation of the two address parts to a linear address, a 32-bit base address is added, to obtain the effective memory address. The base address typically is the starting address of the data array referenced by this GAG. Finally, the bus interface handles the protocol for the actual data transfers between the data memory and the data manipulating devices, using the effective memory address to access the data memory. A block diagram (See Memory address generator of the MoM-3 generic address generator.) illustrates these tasks, as they are performed in the Memory Address Generator. The AIRport register has been introduced for two reasons. First, the instructions in the Address Instruction RAM (AIR) are only 16 bits wide, so that the AIRport register serves as an interface register, to transfer two instructions with one configuration data transfer. Second, to the processor which downloads the parameters, the Address Instruction RAM is hidden behind the AIRport register and consumes only a single address. The programming model corresponds to a shift register, allowing to write a complete instruction stream to the same AIRport address during configuration. This can be done efficiently using block transfers. This concept improves the scaleability of the GAG, because different sizes of the Address Instruction RAM do not require a different layout of configuration register addresses, but only a different length of the block transfers. This is true for the number of instructions that can be stored as well as for the format of the instructions (32-bit instructions would allow for a larger offset range, for example). The resulting changes to the download software can easily be parameterized, whereas the processor interface remains unchanged for all these variations of Memory Address Generators.

Comments on the MoM-3 Address Generator

The Map-oriented Machine 3 (MoM-3) address generator has been developed from the MoM-2 address generator (See The Address Generator of the Map-oriented Machine 2). As well as in the MoM-2 a 2-level address generation using the slider method See [HHW90] is performed. Additionally the MoM-3 utilizes multiple address generators operating in parallel. This accelerates address generation and enables multiple scan windows operating in parallel. Further the MoM-3 address generator supports a flexible low level sequencing, which is needed for the data scheduling performed by the DPSS See [Kre96]. The overall MoM-3 architecture See [Rei99] will be introduced in See The Map-oriented Machine 3. A detailed explanation of the data sequencer of the MoM-3 is given in See [Web93] or See [HR95]. The research presented in this thesis was started with the completion of the MoM-3 implementations. Therefore the MoM-3 data sequencer can be seen as the predecessor of this work. For this reason in this subsection some detailed critique on the MoM-3 data sequencing is given.

The address generation method is optimized to generate data accesses to 2-dimensional data sets. While the address generation hardware is optimized to generate such access sequences, it doesn't exploit 2-dimensional memory organization for access optimizations like e.g. interleaved memory. The only performed memory access optimization is the cache like exploitation of scan window overlappings.

The main address generation is performed by the GAGs. Because the GAGs can't change their parameter set during runtime, they can only perform two nested loops per dimension. To perform more complex scan pattern (e.g. the zig-zag enumeration step of the JPEG image compression method), the GAGs must be stopped and reprogrammed by the software based Instruction Sequencer. Some complex address calculations are completely performed by the Instruction Sequencer. This inflexibility of the GAG causes also, that regular scan windows can't be read generically by the GAG, but must also be read by the MAG. This causes a high configuration overhead for such applications.

After address generation the 2-dimensional address is mapped to the 1-dimensional address space of the linear physical memory. Instead of using a flexible mapping scheme like e.g. the VSP in See The Video Signal Processor, a multiplexer selects between four fixed mappings (see Memory Address Generator at See The Memory Address Generator). While the MoM-3 implementation of this 4 to 1 multiplexer needs more hardware as an optimized barrel shifter, this mapping scheme wastes also a lot of data memory for many applications.

The bus oriented implementation of the MoM-3 causes a serious bus bottleneck. All addresses and data must be transferred via the single MoMbus and all data is stored in the same memory (see See The Map-oriented Machine 3). Since the generic address generation method is quite fast, only one GAG is really working at a time and the others are waiting with a bus request.

While parallel GAGs could logically work concurrently, this coarse grain parallelism is also often impossible because of the data dependencies of the application. Usually the data needed at a time is determined by a single scan window and not by multiple scan windows. As a result most applications don't exploit the possible parallelism of multiple scan windows.

In fact the MoM-3 data sequencer has a big hardware overhead, since only one GAG is working at a time. This overhead could be alleviated for some applications by having multiple datapaths for address and data as briefly considered in See [HKR95a]. In some cases parallelism could be exploited.

In See [Rei99] a GAG architecture, with exchangeable programming, has been briefly considered for the MoM-3. This architecture would avoid the hardware overhead of the implemented MoM-, by programming a single GAG with the parameters for several video scans. During execution the suggested GAG would switch between the configurations.

TMS320C54x Key Features

The CPU (see See The Texas Instruments TMS320C54x DSP architectural overview.) is an advanced multibus architecture with one program bus, three data buses, and four address buses. It includes a 40-bit arithmetic logic unit (ALU), including a 40-bit barrel shifter and two independent 40-bit accumulators. Additionally a 17-bit x 17-bit parallel multiplier is coupled to a 40-bit dedicated adder to perform a non pipelined single-cycle multiply/accumulate (MAC) operation. Furthermore the CPU includes a compare, select, store unit (CSSU) to perform add/compare selection operations as needed in the Viterbi operator. An exponent encoder is integrated to compute the exponent of a 40-bit accumulator value in a single cycle.

To perform data sequencing two dedicated address generators are available, including eight auxiliary registers and two auxiliary register arithmetic units.

Separate program and data spaces allow simultaneous access to program instructions and data, providing a high degree of parallelism. For example, three reads and one write can be performed in a single cycle. Instructions with parallel store and application-specific instructions fully utilize this architecture. In addition, data can be transferred between data and program spaces. Such parallelism supports a powerful set of arithmetic, logic, and bit-manipulation operations that can all be performed in a single machine cycle. Also, control mechanisms to manage interrupts, repeated operations, and function calling are included.

Memory Organization

The TMS320C54x has 192K words x 16-bit addressable memory space (64K-words program, 64K-words data, and 64K-words I/O). Further it integrates several types of on-chip memory:

• 2K, 20K or 32K-words program ROM,
• 0, 8K or 16K-words program/data ROM,
• 5K, 6K, 8K, or 10K-words dual-access RAM (DARAM), and
• 0 or 24K-words single-access RAM (SARAM).

The internal RAMs can be configured as data memory or as program/data memory. The DARAM can be accessed twice per machine cycle, the CPU can read and write to a single block in the same cycle.

Data Addressing

The TMS320C54x has an independent data-address generation logic (DAGEN, see also See The Texas Instruments TMS320C54x DSP architectural overview. and See [TI97]). The CPU offers seven basic data addressing modes. During the execution of instructions using direct, indirect, or memory-mapped register addressing, the DAGEN computes the addresses of data-memory operands. The basic addressing modes are:

• Immediate addressing uses the instruction to encode a fixed value.
• Absolute addressing uses the instruction to encode a fixed address.
• Accumulator addressing uses the accumulator A to access a location in program memory as data.
• Direct addressing uses seven bits of the instruction to encode the lower seven bits of an address. The seven bits are used with the data page pointer (DP) or the stack pointer (SP) to determine the actual memory address.
• Indirect addressing uses the auxiliary registers to access memory.
• Memory-mapped register addressing uses the memory-mapped registers without modifying either the current DP value or the current SP value.
• Stack addressing manages adding and removing items from the system stack.

In the following sections the three addressing modes performed by the DAGEN are explained in detail.

The Direct Data Addressing

In direct addressing mode (see See TMS320C54x direct addressing block diagram.), the instruction contains the lower seven bits of the data-memory address (dma). The 7-bit dma is an address offset that is combined with a base address, with the data-page pointer (DP), or with the stack pointer (SP) to form a 16-bit data-memory address. Using this form of addressing, any of 128 locations can be accessed in random order without changing the DP or the SP.

Either DP or SP can be combined with the dma offset to generate the actual address. The compiler mode bit (CPL), located in status register ST1, selects which method is used to generate the address:

• When CPL = 0, the dma field is concatenated with the 9-bit DP field to form the 16-bit data-memory address.
• When CPL = 1, the dma field is added (positive offset) to SP to form the 16-bit data-memory address.

The Indirect Data Addressing

In indirect addressing, any location in the 64K-word data space can be accessed via a 16-bit address contained in an auxiliary register. The TMS320C54x has eight 16-bit auxiliary registers (AR0-AR7). Indirect addressing is used mainly when there is a need to step through sequential locations in memory in fixed-size steps.

When memory is addressed with indirect addressing, the auxiliary register and the address can be optionally modified by a decrement, an increment, an off-set, or an index. Special modes offer circular and bit-reversed addressing. A circular buffer size register (BK) is used with circular addressing. The AR0 register is used for indexed and bit-reversed addressing modes in addition to being used to point to memory as the other auxiliary registers do.

Indirect addressing is flexible enough not only to read or write a single 16-bit data operand from memory with one instruction, but also to access two data-memory locations with one instruction. Accesses of two data-memory locations include reads of two independent memory locations, reads and writes of two consecutive memory locations, and a read of one memory location combined with a write to a memory location.

Two auxiliary register arithmetic units (ARAU0 and ARAU1) operate on the contents of the auxiliary register. The ARAUs perform unsigned, 16-bit auxiliary register arithmetic operations. The auxiliary registers can be:

• Loaded via an immediate value.
• Loaded via the data bus by writing to the memory-mapped auxiliary registers.
• Modified by the indirect addressing field of any instruction that supports indirect addressing.
• Modified by the modify auxiliary register instruction.
• Used as loop counters.

See TMS320C54x indirect addressing block diagram for dual data-memory operands. shows the ARAUs used to generate an address in the indirect addressing mode using a single data-memory operand. As the figure shows, the main components used for address generation in indirect addressing are the auxiliary register arithmetic units (ARAU0 and ARAU1) and the auxiliary registers (AR0-AR7).

Dual data-memory operands addressing is used for instructions that perform two reads or a single read and parallel store at the same time. These instructions operate all in indirect addressing mode only. If the source operand and the destination operand point to the same location, in instructions with a parallel store, the source is read before writing to the destination.

The Memory-Mapped Register Data Addressing

Memory-mapped register addressing is used to modify the memory-mapped registers without affecting either the current data-page pointer (DP) value or the current stack-pointer (SP) value. Because DP and SP do not need to be modified in this mode, the overhead for writing to a register is minimal. Memory-mapped register addressing works for both direct and indirect addressing. Addresses are generated by:

• Forcing the nine most significant bits (MSBs) of data-memory address to 0, regardless of the current value of DP or SP when direct addressing is used.
• Using the seven LSBs of the current auxiliary register value when indirect addressing is used.

The Target Architecture Styles

An alternative to a fully programmable memory management unit10 (MMU) architecture is a dedicated architecture that benefits from customized implementation possibilities. To alleviate the interconnection overhead in terms of delay, power and area associated with the routing between the underlying (hierarchical) memory architecture and the MMU, a distributed address unit architecture is then selected.

The customized MMU architecture (cMMU) is based on many small application-specific Address Calculation Units (ACUs) which are controlled by a hierarchical (master/slave, global/local) hardwired controller. In the most straightforward and usually inefficient case, this leads to a direct mapping of each individual array index reference onto dedicated hardware, e.g See [LMW91]. This strategy can also lead to a large cost overhead if not executed well, although in this case the overhead only depends on the overall contribution of the local cost in the address generators and not on the program ROM and associated decoders.

For some applications it is still desirable to provide some (re)programming capabilities. In this case, a combination of programmable and customized ACUs is required for flexibility.

For indexed addressing, a fixed storage ordering is selected when multidimensional signals are presented. Typically this order is first optimized for size reduction by so called in-place mapping See [GCM97]. Then, an address expression (AE) for every manifest index expression of a signal located in a linearly addressed RAM is generated.

For custom processors, two clearly differing target architecture styles can be identified: incremental address generation unit (iAGU) and custom address calculation unit (cACU) (see See Adopt target architecture styles for custom processors: (a) example application with index function, (b) custom address calculation unit (cACU), and (c) incremental address generation unit (iAGU).).

In the iAGU case, Address Sequences (ASs) are generated and realized as a counter modified by a two- or multi-level logic filter. This style requires an explicit expansion of the AEs, so it can be applied only to manifest (compile-time known) accesses. Counter based architectures require two control lines to obtain the next memory address, and to set the addressing unit to an initial state. These lines are generated from the global/master cMMU controller.

In the cACU case, the AEs are realized as Application-Specific Units (ASUs) with custom arithmetic building blocks selected from a library. Use is made of a subset of the Cathedral-3 methodology for lowly multiplexed custom data-path synthesis See [NGC91]. This architecture style has been first proposed in an environment as an alternative to the iAGU style for the synthesis of address generators See [MCM94] See [MCJ96]. For cACUs, the complete set of iterator states must be supplied by a local controller. Also data-dependent values can be fed into the inputs of the ASU thus both run-time and manifest data dependencies can be supported.

Selection of Incremental Address Generation Units

For custom targets, the least expensive solution in area and power can be to map the AS onto an iAGU rather than a cAGU. This decision depends heavily on regularity properties of the target AS See [MKC97], and these properties can very accurately predict the need for extra logic between the counter and the address port bits.

Selection of Custom Address Calculation Units

Many other ASs with irregular access patterns are much more area hungry, depending heavily on the sequence length. In this cases, the same AE can be efficiently implemented using a cACU, thus having up to one order of magnitude in combinatorial area.

This trade-off is for power and area since a regular access pattern implemented using a counter has potentially much less toggle activity than if implemented using a cACU. The opposite is true for an irregular pattern.

Mapping of Custom Address Calculation Units

The methodology used for the mapping is not based on performing scheduling first, followed by the traditional functional-unit, mux and register allocation See [FPC90]. In See [MCJ96] has been shown that it is better to first identify similar clusters of expressions (arithmetic of addresses and conditions) in the algorithm. The goal of this phase is to obtain an optimal partitioning of the cACU in ASUs. After cACU partitioning, further regularity improvement between the shared clusters See [JCM96] aiming to decrease the multiplexing overhead is possible by applying local scope algebraic transformations. After that it is mapped by a commercial behavioral synthesis tool.

Synthesis of Incremental Address Generation Units

For iAGUs, also AE multiplexing possibilities are provided but at the sequence level by expanding the AE at compile time. In this way an interleaved (time-shared) AS version of the multiplexed AEs can be obtained.

Again, system level similarities (this time over the ASs) are needed to select the best sharing possibilities. Heuristic measures based on first and second order differences between pairs of AS values are sufficient See [MCM94]. The result is a partitioned iAGU architecture in terms of counters and look-up tables.

After counter sequence sharing decisions, the characteristics of the counter (i.e., counter modulo value, reset and increment state values, etc.) must be defined still for an optimal synthesis of the iAGU unit. Analysis of the (pseudo) periodicity properties of the target AS See [MKC97] See [MCJ97] is exploited. Also, the optimal assignment of counter to address port-bit, detected during the architecture exploration phase is finally incorporated See [GD91].

Once the exact definition of the different iAGUs has been obtained, RT-level merging possibilities amongst the resulting iAGUs can still be explored. Again, similarities across shared ASs are used to decide on the best candidates for the merging of counters and/or look-up tables See [MKC97]. However, as opposed to the sharing step, the search space is now restricted to ASs with common state transitions (same loop scope) to reuse the counters. Performing the final bit-level optimization of the merged look-up tables is left to subsequent logic synthesis.

Functional Description

The Address Sequencer is a 24-bit programmable address generator. As shown in See Intersil HSP45240 address sequencer block diagram., the sequencer consists of 4 functional blocks: the start circuitry, the sequence generator, the crosspoint switch, and the processor interface. The addresses produced by the sequence generator are input into the crosspoint switch. The crosspoint switch maps 24 bits of address input to a 24-bit output. This allows for addressing schemes like "bit-reverse" addressing for FFT's11. A programmable delay block is provided to allow the MSW (most significant word) of the output to be skewed from the LSW (least significant word). This feature may be used to compensate for processor pipeline delay when the sequence generator is configured as two independent 12-bit sequencers. Address Sequencer operation is controlled by values loaded into configuration registers associated with the sequence generator, crosspoint switch, and start circuitry. The configuration registers are loaded through the processor interface.

The Start Circuitry

The Start Circuitry generates the internal START signal which causes the Sequence Generator to initiate an addressing sequence. The START signal is produced by writing the Processor Interface's "Sequencer Start" address (see See [Int97] See [Int98a]), by asserting the STARTlN input, or by the terminal address of a sequence generated under "One-Shot Mode with Restart" (see sequence generator at See The Sequence Generator). A programmable delay from 1 to 31 clocks is provided to delay the initiation of an addressing sequence by delaying the internal START signal.

The Start Circuitry generates the output signal ADDVAL which is asserted when the first valid output address is at the pads. In addition, the Start Circuitry generates the "STARTOUT" signal for multichip synchronization. STARTOUT is only generated when an addressing sequence is started by writing the "Sequencer Start" address of the Processor Interface, or an internal START is generated by reaching the end of an addressing sequence produced by "One-Shot Mode with Restart".

The Sequence Generator

The Sequence Generator is a block oriented address generator. This means that the desired address sequence is subdivided into one or more address blocks, each containing an user defined number of addresses. User supplied configuration data determines the number of address blocks and the characteristics of the address sequence to be generated.

As shown in See Intersil HSP45240 sequence generator block., the Sequence Generator is subdivided into the address generation and control sections. The address generation section performs an accumulation based on the output of MUX1 and MUX2. The control section governs the operation of the multiplexers, enables loading of the Block Start Address register, and signals completion of an address sequence.

An address sequence is started when the control section of the Sequence Generator receives the internal START signal from the Start Circuitry. When the START signal is received, the control section multiplexes the contents of the Start Address Register and a "0" to the adder. The result of this summation is the first address in the first block of the address sequence. This value is stored in the Block Start Address register by an enable generated from the control section, and the multiplexers are switched to feed the output of the Holding and Address Increment registers to the adder. Address generation will continue with the Address Increment added to the contents of the Holding Register until the first address block has been completed.

An address block is completed when the number of addresses generated since the beginning of the address block equals the value stored in the Block Size register. When the last address of the block is generated, BLOCKDONE is asserted to signal the end of the address block (see See [Int98b]). On the following CLK, the multiplexers are configured to pass the contents of the Block Start Address and Block Increment registers to the adder which generates the first address of the next address block. An enable from the control section allows this value to update the Block Start Address register, and the multiplexers are switched to feed the Holding and Address Increment registers to the adder for generation of the remaining addresses in the block.

The address sequence is completed when the number of address blocks generated equals the value loaded into the Number of Blocks register. When the final address in the last address block has been generated, DONE and BLOCKDONE are asserted to signal the completion of the address sequence.

The parameters governing address generation are loaded into five 24-bit configuration registers via the Processor Interface. These parameters include the Start Address, the beginning address of the sequence; the Block Size, the number of addresses in the address block; the Address Increment, the increment between addresses in a block; the Number of Blocks, the number of address blocks in a sequence (minimum 1); the Block Increment, the increment between starting addresses of each block. The loading and structure of these registers is detailed in See [Int97] or See [Int98a].

Three modes of operation may be selected by loading a 6-bit Mode Control register. The three modes of operation are:

• One-Shot Mode without Restart Address generation halts after completion of the user specified address sequence. Address generation will not resume until the internal START signal is generated by the Start Circuitry. When the final address in the final block of the address sequence is generated, both DONE and BLOCKDONE are asserted and the last address is held on OUT0-23 (see See [Int98b]).
• One-Shot Mode with Restart: This mode is identical to One-Shot Mode without Restart with the exception that the Start Circuitry automatically generates an internal START at the end of the user specified sequence to restart address generation. The end of the address sequence is signaled by the assertion of DONE, BLOCKDONE, and STARTOUT as shown in See [Int98b]. In this mode, the first address of the next sequence immediately follows the last address of the current sequence if start delay is disabled.
• Continuous Mode: Address generation never terminates. Address generation proceeds based on the Start Address, Address Increment, Block Size, and Block Increment Parameters. The Number of Blocks parameter is ignored, and the DONE signal is never asserted.

The Mode Control register is also used to configure the Sequence Generator for operation as two independent 12-bit address sequencers. In dual sequencer mode, the adder in the sequence generator suppresses the carry from the 12 LSBs (least significant bits) to the 12 MSBs (most significant bits). With the carry suppressed, two independent sequences may be produced. These 12-bit address sequences may be delayed relative to each other by programming the Mode Control register for a delay up to 7 clocks. This feature is useful to compensate for pipeline delay when using dual sequencer mode to generate read/write addressing. The DLYBLK input can be used to halt address generation at the end of any address block within a sequence. In addition, DLYBLK can be used to delay an address sequence from restarting if asserted at the end of the final address block generated under "One-Shot Mode with Restart". See See [Int98b] for the timing relationship of DLYBLK to the end of the address block required to halt address sequencing.

The Crosspoint Switch

The crosspoint switch is responsible for reordering the address bits output by the sequence generator. The switch allows any of its 24 inputs to be independently connected to any of its 24 outputs. The crosspoint switch outputs can be driven by only one input, however, one input can drive any number of switch outputs. If none of the inputs are mapped to a particular output bit, that output will be "low".

The input to output map is configured through the processor interface. The I/O map is stored in a bank of 24 configuration registers. Each register corresponds to one output bit. The output bit is mapped to the input via a value, 0 to 23, stored in the register. After power-up, the user has the option of configuring the switch in 1:1 mode by using the reset input, "RST". In 1:1 mode the crosspoint switch outputs are in the same order as the input. More details on configuring the switch registers are contained in See [Int97].

Parallelism

All data sequencers presented are designed to operate more or less in parallel to the data manipulations to be performed by an ALU. This is a basic requirement to avoid delays in data manipulations caused by hold up data supply. As already mentioned in before, examples have been published See [PAC97], where a state of the art microprocessors stays idle for 75% of computation time because of outstanding memory requests. If the machine architecture supports concurrent data streams like the VSP (See The Video Signal Processor) and the TMS320C54x (See The Texas Instruments TMS320C54x DSP), the data throughput can be increased significantly. Also the address generator of the MoM-3 (See The Address Generator of the Map-oriented Machine 3) provides multiple generic address generators operating in parallel, but the single MoMbus does not support concurrent data accesses.

Reduction of the Number of Memory Cycles by Reducing Computational Overhead

The number of memory cycles may be reduced by a software to hardware migration of address computation. Software level address generation additionally burdens the memory interface with accesses for instructions, constants and variables to perform the address computation. A hardware level address generator would avoid such memory cycles completely. But since all presented address generators are optimized to reduce the latency of the address generation for a specific application domain, they are not general purpose. The MoM-1 (See The Address Generator of the Map-oriented Machine 1) and the VSP (See The Video Signal Processor) support only video scans. The SMA machine (See The Structured Memory Access Machine), the MoM-2 (See The Address Generator of the Map-oriented Machine 2), the MoM-3 (See The Address Generator of the Map-oriented Machine 3), and the HSP45240 (See The Intersil HSP45240 Address Sequencer) use a backup controller or microprocessor to perform address computations and/or to perform frequent reprogramming for complex address sequences. The TMS320C54x (See The Texas Instruments TMS320C54x DSP) DSP uses its ALU for complex address computations. But powerful address generators should avoid switching the address generation to the ALU even for complex address sequences. Therefore to set-up a high performance address generator, which is also flexible, a novel concept is needed.

The number of memory cycles may also be reduced by identifying and eliminating multiple accesses to the same data:

Avoiding Multiple Accesses to the Same Data

Specialized data sequencers have also to potential to accelerate memory accesses by access optimizations. The MoM-2 and MoM-3 data sequencers (See The Address Generator of the Map-oriented Machine 2 and See The Address Generator of the Map-oriented Machine 3) perform window based memory accesses, which can be seen as a cache like optimization. Data accesses are assigned to computation steps. Algorithms, which access the same data in successive computation steps, are optimized in a way that this data is only read once and stored in a smart interface. This optimization works only for local data.

Other address generators (MoM-1 and HSP45240) do not support any access optimization. The microprocessor examples (SMA machine, VSP, and TMS320C54x) may perform the typical access optimizations known from other microprocessors (e.g. hierarchic memory).

Reduced Configuration Code Size

The generic approach of most of the presented address generators reduces also the reconfiguration load. The presented hardware examples are configured by a few parameters and addresses are generated according to a generic model.