というわけで @h_kenken 氏から指摘があって実はシリアルの結線が間違っていたことがわかり、
ちゃんと配線するとシリアル入出力ともにちゃんと動きました。

そしてboot時に ^C を押すとあっさり RedBoot に落ちれました。あぁ…

++Initializing SPI-NOR flash...
JEDEC ID: 0xbf:0x25:0x41
SPI NOR: block_size=0x10000, blocks=0x20, start=0x00000000, end=0x00200000
... Read from 0x1fef0000-0x1ff00000 at 0x00080000: .
... Read from 0x1fee3000-0x1fee4000 at 0x0008f000: .
PMIC ID: 0x000045d0 [Rev: 2.0a]
[Warning] FEC not connect right PHY: ID=ffffffff
No network interfaces found

Reset reason: Power-on reset
fis/fconfig from SPI-NOR
Boot switch: INTERNAL
        EXPANSION: SPI-NOR: eCSPI1

RedBoot(tm) bootstrap and debug environment [ROMRAM]
Non-certified release, version FSL 200929 - built 21:29:52, Sep  9 2009

Platform: MX51 SHARP PC-Z1 [1.000] (Freescale i.MX51 based) PASS 2.0 [x32 DDR]
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2003, 2004, 2005, 2006 eCosCentric Limited

RAM: 0x00000000-0x1ff00000, [0x000a7bc8-0x1fee1000] available
FLASH: 0x00000000 - 0x200000, 32 blocks of 0x00010000 bytes each.
== Executing boot first script abort ==
== Executing boot script in 0.000 seconds - enter ^C to abort
RedBoot> ^C
RedBoot> ^C
RedBoot> ^C
RedBoot> ^C
RedBoot> help
Setup/Display clock
Syntax:
   clock [<core clock in MHz> :<DDR clock in MHz>] 

   Examples:
   [clock]         -> Show various clocks
   [clock 665]     -> Core=665  
   [clock 800:133]  -> Core=800  DDR=133 
   [clock :166]   -> Core=no change  DDR=166 

console ON/OFF
   console <ON/OFF>
EXT2 debug mode
   dbg_e2fs <mode>
EXT2 sector read
   dbg_e2fs_read sector
SDRAM calib
   ddr2calib  
L2 cache
   L2 [ON | OFF]
AD input
   adin <channel> {-l low-limit} {-u up-limit}
Manage aliases kept in FLASH memory
   alias name [value]
Set/Query the system console baud rate
   baudrate [-b <rate>]
Manage machine caches
   cache [ON | OFF]
Display/switch console channel
   channel [-1|<channel number>]
Compute a 32bit checksum [POSIX algorithm] for a range of memory
   cksum -b <location> -l <length>
Display disks/partitions.
   disks 
Display (hex dump) a range of memory
   dump -b <location> [-l <length>] [-s] [-1|-2|-4]
Execute an image - with MMU off
   exec [-w timeout] [-b <load addr> [-l <length>]]
        [-r <ramdisk addr> [-s <ramdisk length>]]
        [-c "kernel command line"] [-t <target> ] [<entry_point>]
Manage FLASH images
   fis {cmds}
Manage configuration kept in FLASH memory
   fconfig [-i] [-l] [-n] [-f] [-d] | [-d] nickname [value]
blow some fuses
   fuse_blow <bank> <row> <value>
read some fuses
   fuse_read <bank> <row>
Execute code at a location
   go [-w <timeout>] [-c] [-n] [entry]
Help about help?
   help [<topic>]
Display command history
   history 
Set/change IP addresses
   ip_address [-b] [-l <local_ip_address>[/<mask_len>]] [-h <server_address>]
Load a file
   load [-r] [-v] [-z to swap endianness on 16 bit] [-d] [-h <host>] [-p <TCP port>][-m <varies>] [-c <channel_number>] 
        [-b <base_address>] <file_name>
Compare two blocks of memory
   mcmp -s <location> -d <location> -l <length> [-1|-2|-4] {-f}
Copy memory from one address to another
   mcopy -s <location> -d <location> -l <length> [-1|-2|-4]
Fill a block of memory with a pattern
   mfill -b <location> -l <length> -p <pattern> [-1|-2|-4]
Read/Write MMC/SD card
   sdhc <ram-addr> <flash-addr> <len-bytes> <r/w>
Utility function to NAND flash using raw address
   nand {cmds}
Network connectivity test
   ping [-v] [-n <count>] [-l <length>] [-t <timeout>] [-r <rate>]
        [-i <IP_addr>] -h <IP_addr>
Read/Write internal PMIC register
   pmic <reg num> [value to be written]
Power OFF
   poweroff 
Reset the system
   reset 
start from SD
   sdstart <-c "kernel command line"> <-f conf> <-s slot1/2>
Read/Write/Erase SPI NOR flash
   spiflash <ram-addr> <flash-addr> <len-bytes> <r/w/e>
Display RedBoot version information
   version 
Display (hex dump) a range of memory
   x -b <location> [-l <length>] [-s] [-1|-2|-4]
Enable one flash media for Redboot
   factive [MMC|SPI]
GPIO read
   gpio <group>
GPIO write
   gpiow <group> <offset> <data>
read info from Serial-Flash
   inforead -o <offset> {-l <length> [-1|-2|-3|-4]} {-b <ram-address>}
wtite info to Serial-Flash
   infowrite -o <offset> <value> {-l <length> [-1|-2|-3|-4]} {-b <ram-address>}
IOMUX read
   iomux <offset> [<length>]
IOMUX write
   iomuxw <offset> <data>
Update Redboot with currently running image
   romupdate 
Run an image at a location with MMU off
   run [<virtual addr>]
SDRAM test
   sdramtest <start-address> <length>
SD/MMC read image from specify address
   sdread -f <sd address>/-o <sd sector> -b <ram address> -l <length> [-s <slot 1/2>]
SD/MMC register DISK
   sdregister <slot 1/2>
SD/MMC update Redboot by specify address
   sdupdate <address> [-p <forcepart>] [-c <slot 1/2>]
SD/MMC write image by specify address
   sdwrite -f <sd address>/-o <sd sector> -b <ram address> -l <length> [-s <slot 1/2>]
Set FEC MAC address in Fuse registers
   set_fecmac [0x##:0x##:0x##:0x##:0x##:0x##]
Serial-Flash update Redboot by specify address
   sromupdate <address>
RedBoot> clock

PLL1          PLL2              PLL3
========================================
800000000       665000000       216000000

CPU           AHB               IPG             EMI_CLK
========================================================
800000000       133000000       66500000        200000000

NFC           USB               IPG_PER_CLK
========================================
26600000        54000000        24000000

UART1-3       SSI1              SSI2            SPI
========================================================
66500000        54000000        54000000        54000000

以下、デフォルト状態のboot script

Boot first script:
.. adin 5 -l 6800

Boot script:
.. nand init
.. nand read -f 0x00000000 -b 0x00800000 -l 0x00500000
.. exec -b 0x00800000 -l 0x00500000 -c "noinitrd console=ttymxc0,115200 ubi.mtd=4 root=ubi0:rootfs rw rootfstype=ubifs"

というわけで、fcofnig で Boot script を false にしたり、中身を sdstart にしたりすると、
デフォで RedBoot が上がってきたりSDカードからBootしたりできます。これでちょっとは楽になる?