Blackfin arch: Fix bug - when to rmmod the L1_module, it stucks and then reboot the board.

Fix this by correcting the wrong pointer

Signed-off-by: Sonic Zhang <sonic.zhang@analog.com>
Signed-off-by: Bryan Wu <cooloney@kernel.org>