izmit escort escort ankara adana escort

 FAQ •  Search •  Register •  Login 

All times are UTC + 1 hour [ DST ]




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Fri May 26, 2017 10:45 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Hi all,
I have custom board with AT91SAM9G25. Previously I used kernel 3.13.5 with no reboot problems (but problems with serial line errors). After update to 4.1.0 serial line (and all other) was OK, excepting problems with software reboot. Sometimes (1x of 15-50 tries) board hangs instead of reboot.
I have added debug printk to /drivers/power/reset/at91-reset.c
Code:
static int at91sam9g45_restart(struct notifier_block *this, unsigned long mode,
             void *cmd)
{
  printk("at91sam9g45_restart\n");//MM
  asm volatile(
    /*
     * Test wether we have a second RAM controller to care
     * about.
     *
     * First, test that we can dereference the virtual address.
     */
    "cmp  %1, #0\n\t"
    "beq  1f\n\t"

    /* Then, test that the RAM controller is enabled */
    "ldr  r0, [%1]\n\t"
    "cmp  r0, #0\n\t"

    /* Align to cache lines */
    ".balign 32\n\t"

    /* Disable SDRAM0 accesses */
    "1: str %3, [%0, #" __stringify(AT91_DDRSDRC_RTR) "]\n\t"
    /* Power down SDRAM0 */
    " str %4, [%0, #" __stringify(AT91_DDRSDRC_LPR) "]\n\t"
    /* Disable SDRAM1 accesses */
    " strne %3, [%1, #" __stringify(AT91_DDRSDRC_RTR) "]\n\t"
    /* Power down SDRAM1 */
    " strne %4, [%1, #" __stringify(AT91_DDRSDRC_LPR) "]\n\t"
    /* Reset CPU */
    " str %5, [%2, #" __stringify(AT91_RSTC_CR) "]\n\t"

    " b .\n\t"
    :
    : "r" (at91_ramc_base[0]),
      "r" (at91_ramc_base[1]),
      "r" (at91_rstc_base),
      "r" (1),
      "r" cpu_to_le32(AT91_DDRSDRC_LPCB_POWER_DOWN),
      "r" cpu_to_le32(AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST)
    : "r0");

  printk("at91sam9g45_restart asm done \n");//MM
  return NOTIFY_DONE;
}


and only first printk line prints (it is normal),but no other printout from ROMboot, like in the normal response if reboot works.
Yesterday I have tried also actual 4.4.68 kernel from https://github.com/linux4sam/linux-at91.git
I did not added full support to our board, only partial, but tried sw reboots, and also this kernel hangs in the same point. So I would prefer to fix my 4.1.0 version...
Is possible to do software reset in other way? (I can not use watchdog, because it is disabled in AT91Bootstrap which can be only flashed via SAM-BA and I have many boards in fields - it is problem to upgrade them remotely, if reboot can cause hang...)
I have almost no experiences with gdb, can you help me please, how to debug and fix this problem?
Thanks

successfull reboot outoput to serial line (on kernel 4.1.0 with some additional printks):
Code:
The system is going down NOW!
Sent SIGKILL to all processes
Requesting system reboot
[   28.604000] reboot: Restarting system
[   28.604000] Local irq disable
[   28.604000] smp_send_stop
[   28.604000] do kernel restart (null)
omBOOT.604000] at91sam9g45_restart
 ba_offset = 0xb ...


AT91Bootstrap 3.5.3 (▒t ▒ec  9 10:06:17 CEST 2015)



unsuccessfull reboot outoput to serial line (on kernel 4.1.0 with some additional printks):
Code:
[   26.780000] UBIFS (ubi0:0): background thread "ubifs_bgt0_0" stops
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[   28.708000] reboot: Restarting system
[   28.708000] Local irq disable
[   28.708000] smp_send_stop
[   28.708000] do kernel restart (null)
[   28.708000] at91sam9g45_restart

and hang...


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Wed May 31, 2017 2:13 am 
Offline

Joined: Thu Apr 19, 2007 10:15 pm
Posts: 1470
Location: USA
You're rather sloppy/vague in your problem description.
How is this random event of "1x of 15-50 tries" distributed?
Is it just one board, or occurs with every board, or ???

Regards


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Tue Jun 13, 2017 8:56 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Sorry,
I meant one restart in 15-50 "reboot" commands will fail... It seems to be board indipendent (I have tried it on three different custom boards, on six boards at all). My firmware test script logs-in to board (via ssh), and proceeds 'reboot' command (and measures time between reboots). I have also connected serial console to debug the last kernel outputs. After some number of successfull reboots, system hangs during the last phase of reboot (assembler).
Sorry for may bad English.
Regards


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Wed Jun 14, 2017 2:10 pm 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Unfortunatelly, I have problems with assembler code. This is previous -not hanging- kernel (3.13.5) assembler code for at91sam9g45_restart (it was in the .S file):
Code:
      .arm
/*
 * at91_ramc_base is an array void*
 * init at NULL if only one DDR controler is present in or DT
 */
      .globl  at91sam9g45_restart

at91sam9g45_restart:
      ldr r5, =at91_ramc_base   @ preload constants
      ldr r0, [r5]
      ldr r5, [r5, #4]      @ ddr1
      cmp r5, #0
      ldr r4, =at91_rstc_base
      ldr r1, [r4]

      mov r2, #1
      mov r3, #AT91_DDRSDRC_LPCB_POWER_DOWN
      ldr r4, =AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST

      .balign 32        @ align to cache line

      strne r2, [r5, #AT91_DDRSDRC_RTR] @ disable DDR1 access
      strne r3, [r5, #AT91_DDRSDRC_LPR] @ power down DDR1
      str r2, [r0, #AT91_DDRSDRC_RTR] @ disable DDR0 access
      str r3, [r0, #AT91_DDRSDRC_LPR] @ power down DDR0
      str r4, [r1, #AT91_RSTC_CR]   @ reset processor

      b .


I can see that there is swapped disabling DDR0 and DDR1 memories and little bit different comparison at the begginning... Can somebody compare it with my first post and try to make a suggestion how to modify "c-like" assembler in the first post according to this assemmbler, to try if it is solution?
If it is possible, please post this "c-like" code here (I am affraid, that I am not able to modify it by myself).
Thanks.
Martin


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Wed Jun 14, 2017 7:03 pm 
Offline

Joined: Wed Aug 19, 2015 9:44 pm
Posts: 53
misarm wrote:
Unfortunatelly, I have problems with assembler code. This is previous -not hanging- kernel (3.13.5) assembler code for at91sam9g45_restart (it was in the .S file):
Code:
      .arm
/*
 * at91_ramc_base is an array void*
 * init at NULL if only one DDR controler is present in or DT
 */
      .globl  at91sam9g45_restart

at91sam9g45_restart:
      ldr r5, =at91_ramc_base   @ preload constants
      ldr r0, [r5]
      ldr r5, [r5, #4]      @ ddr1
      cmp r5, #0
      ldr r4, =at91_rstc_base
      ldr r1, [r4]

      mov r2, #1
      mov r3, #AT91_DDRSDRC_LPCB_POWER_DOWN
      ldr r4, =AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST

      .balign 32        @ align to cache line

      strne r2, [r5, #AT91_DDRSDRC_RTR] @ disable DDR1 access
      strne r3, [r5, #AT91_DDRSDRC_LPR] @ power down DDR1
      str r2, [r0, #AT91_DDRSDRC_RTR] @ disable DDR0 access
      str r3, [r0, #AT91_DDRSDRC_LPR] @ power down DDR0
      str r4, [r1, #AT91_RSTC_CR]   @ reset processor

      b .


I can see that there is swapped disabling DDR0 and DDR1 memories and little bit different comparison at the begginning... Can somebody compare it with my first post and try to make a suggestion how to modify "c-like" assembler in the first post according to this assemmbler, to try if it is solution?
If it is possible, please post this "c-like" code here (I am affraid, that I am not able to modify it by myself).
Thanks.
Martin


At least the .balign directive is suspect. It will create a 0 - 30 byte hole in the code, and in 15 cases of 16 your code will get into never-never land there.

Try this instead:

b aligned
.balign 32
aligned:


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Thu Jun 15, 2017 8:31 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Thanks tsvoipio,
I have tried to add
Code:
b aligned
.balign 32
aligned:

instead of
Code:
.balign 32

but problem still persists (after 37 successfull attempts system hung and didn't reboot in the same way).
Maybe it is not in the at91sam9g45_restart asm code, but in some other condition/state processor is in, before reboot.
I have unsuccessfully tried to rewrite older kernel's linux-3.13.5/arch/arm/mach-at91/at91sam9g45_reset.S to newer kernel's /drivers/power/reset/at91-reset.c, but i am doing some mistake in asm->'c-like' asm conversion (kernel ends with panic instead of reboot).


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Thu Jun 15, 2017 3:08 pm 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
I studied manual and I have added AT91_RSTC_EXTRST to AT91_RSTC_CR to see on external reset pin if reset is done (NRESET pin goes low for some time), and in case of reboot fail IS NOT (only in case of successfull reboot).
So I changed end of c asm function in such (dirty) way:
Code:
    /* Reset CPU */
    " str %5, [%2, #" __stringify(AT91_RSTC_CR) "]\n\t"
    " b .\n\t"

to
Code:
    /* Reset CPU */
    "reset:\n\t"
    " str %5, [%2, #" __stringify(AT91_RSTC_CR) "]\n\t"
    " b reset\n\t"
    " b .\n\t"

and I have now 200+ successfull reboot tries (i hope it is fixed now - I have no problem till now).
If somebody knows, what could happened,or how to debug it more, please feel free to suggst better solution.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Mon Jun 19, 2017 8:00 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
After stronger testing, I see, that problem still persists, only is occuring with less probability (one of 35 to 200 tries). I tried to modify asm in several ways, but no luck.
After hang, system can be rebooted by external reset (if I pull it to ground) and I am suspecting, that asm reset instruction is not proceeding.
Code:
str %5, [%2, #" __stringify(AT91_RSTC_CR) "]\n\t"

I don't know, how to continue, to debug and avoid this problem completely...


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Mon Jun 19, 2017 2:02 pm 
Offline

Joined: Wed Aug 19, 2015 9:44 pm
Posts: 53
Does anything ensure that there are no interrupts between the RAM switch-off and the reset instruction?

The interrupt handler will end in never-never land if it is attempting to run from the RAM which is switched off.

I just wonder why the DDR is switched off for a reboot.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Mon Jun 19, 2017 2:24 pm 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Thank you,
There is a notice
Code:
* unless the SDRAM is cleanly shutdown before we hit the
* reset register it can be left driving the data bus and
* killing the chance of a subsequent boot from NAND

We are booting from NAND, so I don't want to disable it forever, but I can check if it will be better for now...

I will try also to check interrupts...


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Tue Jun 20, 2017 7:29 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
Thanks,
after skipping RAM shutdown, I had 250+ successfull tries and still no hang, I will continue today.


Top
 Profile  
Reply with quote  
 Post subject: Re: Problem on AT91SAM9G25 - randomly is not rebooting
PostPosted: Wed Jun 21, 2017 8:04 am 
Offline

Joined: Wed Oct 03, 2007 9:31 am
Posts: 18
Location: EUROPE
I have successfully rebooted 1150 times. We don't use shared pins for DDR and NAND, so I hope I can savely disable RAM shutdown.
Thank you (especially tsvoipio). for me is problem solved now.
MM


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC + 1 hour [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron