Picaxe web pages

julianE

Senior Member
Hello, I have been using a combination of picaxe and raspberry pi nano for a while now, it works great but for one major issue. Raspberry Pi uses an SD card as a hard drive and the SD card fails in a little less than a year. It's not an issue for me I just burn a backup of the OS and all is well again. I am planning on setting up a system for family and it would be annoying for the Pi to stop working. On the other hand Picaxe is rock solid I have had projects running for multiple years without issues. I have used ESP-8266 before and tho succesful it was a bit fiddly. I'm interested in what works well and is simple and reliable, I don't need any complicated web pages plain HTML is good enough. I'm not averse to programming in Python or even arduino IDE, I'll be reading up on ESP32 in the meanwhile. Appreciate any feedback, thank you.
 

PhilHornby

Senior Member
To stop your SD cards being destroyed, you probably need a Raspberry Pi OS configuration that is not forever writing log files to them. You'd think there'd be a minimal distribution somewhere, that came with everything OFF by default, but I've not found one.
(PS I've hated unix and all its descendants, since I first clapped-eyes on it 1987 - I thought I was going to have to find a new career, but fortunately, nothing ever came of it :unsure:)

I've previously implemented a Webserver using a Picaxe 14M2, via a (now obsolete) TLN13UA06 module, using inspiration from here (and later derivatives). The main issue on the Picaxe, is that a) You have to implement the whole HTTP: protocol and b) there's never enough memory! To get round the second issue, I used both slots, the TABLE data, EEPROM data and sent some of it encoded in hserout text strings!

Since the TLN13UA06 is no longer available, I've been meaning to revisit the Wifi part using an ESP8266 (haven't got as far as the ESP32 yet). The "AT" firmware that the ESP8266 comes pre-loaded with, is a real waste of the module's potential - you really want it to implement the actual Webserver. (I've done the latter on a modified Sonoff wifi switch, but just haven't got round to coming up with a standardised method of swapping data with a Picaxe).
 

Flenser

Senior Member
Another approach is to run Linux from a filesystem in RAM. Linux is installed on the hard drive/SSD/SD card and as a part of it's startup process it creates a RAM drive, copies the Linux distribution onto it and then boots Linux from that RAM drive.
I don't have any experience with the Raspberry Pi and I don't know whether there are already any of this type of Linux distribution available for it.
This Wikipedia article lists several of the type of Linux distribution that I'm talking about https://en.wikipedia.org/wiki/List_of_Linux_distributions_that_run_from_RAM.
As you will see the distributions in this list are all built to be run on PCs as they boot from CD, DVD, HDD as well as USB flash drive and some still from a Floppy disk.
 

julianE

Senior Member
i'm thinking pi can boot from an external USB SSD hard drive but now we are piling on expenses, i have at least a dozen nano boards in stock, it's really an impressive board for $5. I did buy some high quality SD cards to see if they will last longer and am trying one now. the way the SD cards fail is intermittent, i had some refuse to boot while others would run but with some functions failing.
 

Flenser

Senior Member
JulianE,
I did not make myself clear enough. I'm not suggesting that you boot your Pi's from a USB SSD hard drive. I'm suggesting that you search the web to see if anyone has built a Linux distribution for the the Pi that boots from the standard SD card on the Pi to run from a RAM drive.
That wikipedia article only lists PC distributions and I posted it only to show you that you can run Linux from a RAM drive (and so that you could try one of them out on a PC to see how they work, if you wanted to.)
I do not know if any such distribution for the Pi actually exists . It's just one possible solution to your problem of the SD card failing after about a year.
I started running Seti@Home on 26 Jun 1999 (My BOINC stats) using an i386 PC I was given by a mate that did not have a hard drive and I was booting Linux from a 3.25" floppy to run from a RAM disk in this manner.
 

lbenson

Senior Member
I'm interested in what works well and is simple and reliable, I don't need any complicated web pages plain HTML is good enough.
You might want to look into a BASIC implementation on an ESP32 with Annex RDS. There's even a PCB to allow a TFT LCD, SD card & audio, and to make real world connections easy--under $6USD with slow boat shipping.
 

hippy

Technical Support
Staff member
I have been using a combination of picaxe and raspberry pi nano for a while now, it works great but for one major issue. Raspberry Pi uses an SD card as a hard drive and the SD card fails in a little less than a year.
I am not sure what a "Raspberry Pi Nano" would be ?

The question for me would be why the Pi's SD card is getting worn out so rapidly, what you are doing which causes that ?

If it's something you are doing, the way you are doing it, it could just be a matter of doing it some other way.

I'm interested in what works well and is simple and reliable, I don't need any complicated web pages plain HTML is good enough. I'm not averse to programming in Python or even arduino IDE, I'll be reading up on ESP32 in the meanwhile. Appreciate any feedback, thank you.
So what are you wanting, what do you need your system to do ?

For me a Pi would seem the most obvious platform for bridging a PICAXE or microcontroller and the human world of web browsers and the like, a cheap and easily usable replacement for the PC one would normally have chosen for that in the past.
 

PhilHornby

Senior Member
The question for me would be why the Pi's SD card is getting worn out so rapidly, what you are doing which causes that ?
A year's usage, is apparently, not unexpected. The software linked to, is a half-way house to @Flenser's ram disk solution.

For me a Pi would seem the most obvious platform for bridging a PICAXE or microcontroller.
My first I.O.T. project was implemented on a Pi. I decided there were far too many moving parts for my liking and migrated it to a pair of Picaxe 20x2's. The migration took longer than the original project :rolleyes:
 

julianE

Senior Member
Thanks for all the info gents. I'm not doing much of anything with the pi, it collects data from picaxe sensors and serves web pages on the local network using apache on the pi. i have also used thingspeak to receive and display data with picaxe controlling an ESP8266. Doing my research it's perfectly normal for pi SD card to fail after being on for a year. I have had numerous failures, I have a lot of raspberry PI nanos many have a camera attached which does use the SD card a lot. Nano is the least expensive Raspberry Pi to answer Hippy's question. As far as running Linux on internal RAM I'm thinking it would be possible on newer PI but not sure about the low power Nano. A solution I'm toying with, keep it simple type of thing, just have a spare loaded SD card sitting in the box, upon PI failure I can just have the user swap SD cards and bob's your uncle.

Thanks for the ESP32 info lbenson, I am ordering one from amazon comes with an oled built in.

I think the SD card storage is the achilles heel of PI way too unreliable but having a linux machine for less than $11 I really can't complain. At that price I'm thinking you can have a hot standby PI and as one fails the other takes over. BTW, the PI can also program the Picaxe remotely using the write up posted on the forum.

all the best.
 

Flenser

Senior Member
The question for me would be why the Pi's SD card is getting worn out so rapidly, what you are doing which causes that ?
If it's something you are doing, the way you are doing it, it could just be a matter of doing it some other way.
Hippy,
This appears to have been a general issue running Linux on a Pi from a SD Card for a very long time. A web search comes up with articles dating back 10 years. I have two posts to raspberrypi.stackexchange.com that date to 2012 which I'll include later, and another one with recommendations for choosing SD cards for the Pi that was started in 2014 and has been continually updated since then.

julianE,
i couldn't help myself from doing some more research and I've come up with the following links.
These come with the disclaimers that 1) I have no expertise with SD Cards and 2) I have not tested any of these suggestions.

I did buy some high quality SD cards to see if they will last longer and am trying one now.
This could extend the life of your SD Card, and another thing to try is a bigger capacity higher quality SD Card.

This post, which is the one started in 2014, claims that the problem is because cheap SD Cards don't include wear levelLing so the writes to the SD Card eventually wear out some of the blocks, making the card unusable: What are the best SD cards to use in a Raspberry Pi?
and makes two recommendations:
1) Use a more exepensive SD Card that does have wear leveling
2) Use a bigger capacity SD Card than you actually need.

This post from two years ago makes a similar claim that "Vast majority of SD cards do not have wear-leveling" but the person posting this wanted to let people know that "SD cards with wear-leveling used to be expensive, and small in size, and were usually reserved for specialized use (i.e. industrial applications). Today, you can get a 64GB SD card with WL for as low as 15 EUR.": Quick reminder that SD cards with wear-leveling are now fairly common (i.e. WD Purple microSD)
The poster discusses his problems getting WD to give him a simple yes or no answer to whether one of their SD Cards did have wear levelling but, unfortunately, doesn't describe how we might easily find this out for other cards.

This post covers several things but the item I want to draw attention to here is where it points out that the important thing about having extra capacity is the amount of extra free space you get and not the amount of extra total space: How can I extend the life of my SD card?
It gives the example that "if you have a 2 GB SD card with 200 MB free then switching to a 4 GB card will give you 11 times the free space, wear levelling capacity and thus longevity, while switching to a 16 GB card will give you 71 times the free space"

My understanding is that that SSD Hard Drives also use flash chips but they come with wear levelling and are many, many times much larger than SD Cards so while they are are subject to exactly the same wear problem as SD Cards they take much longer to fail.

The last post "How can I extend the life of my SD card?" also gives several suggestions about how to modify your Linux configuration so as to greatly reduce the amount of writes the the OS does to the SC Card:
  • Disable Swap (You only have 512kB RAM on the Pi Nano so this may not be practical)
  • Disabling Journaling on the Filesystem (The downside of doing this is that if the OS crashes then the files on your SD Card could be corrupted. The card would still be OK but you'd still need to re-install Linux)
  • The noatime Mount Flag (Provided that your app does not depend on being able to detect when a file has been udpated I'm not aware of any downside to this)
  • Directories in RAM (You will lose the contents of any directories in RAM when the OS is shutdown or rebooted)
These last options could be more expensive in that they require your time to research and make these OS configuration changes but, on the other hand you could try them on any SD Card to further extend the life of that card.

the way the SD cards fail is intermittent, i had some refuse to boot while others would run but with some functions failing.
Answer 1 in this post states that when SD Cards fail they "just go read-only if you are lucky and die if you are not" which is consistent with the two types of failure that you describe: Tool to monitor SD-Card health
 

PhilHornby

Senior Member
Answer 1 in this post states that when SD Cards fail they "just go read-only if you are lucky and die if you are not"
When the SD card failed in my dashcam, I didn't find out until I tried to look at a recording - the dashcam software simply hadn't noticed that everything it was writing was being discarded. :(
 

lbenson

Senior Member
I've also experienced SD failure with the pi. I think (but couldn't prove) that the latest OS versions are better, but the SD card is still at risk. I looked at some of the options for reducing wear, and it looked like a lot of hoops to jump through. I'd like to see a very straightforward ram disk solution.

@julianE--can you provide a link to this "Raspberry Pi Nano"--in searching, the only thing I found was a Waveshare PiZW clone for $79.
 

hippy

Technical Support
Staff member
Doing my research it's perfectly normal for pi SD card to fail after being on for a year. I have had numerous failures,
this appears to have been a general issue running Linux on a Pi from a SD Card for a very long time.
There have been reports of SD Cards failing since the Pi emerged 10 years ago, but that was mostly corruption and going read only rather than wearing out. Raspberry Pi improved their software quite some time ago to avoid whatever issues were behind that and things have greatly improved.

While it is possible to wear an SD Card out the notion that an SD Card will fail under typical use within a year seems unfounded to me.

Nano is the least expensive Raspberry Pi to answer Hippy's question.
"Nano" doesn't appear to be a Raspberry Pi product at all. The full range has been Pi A, B, A+, B+, 2B, 3B, 3A+, 3B+, 4B, Pi 400, Zero, Zero W, Zero 2W, plus a variety of "CM" Compute Modules.

Perhaps you could post a link to this "Nano" device ?

But it doesn't really matter. It appears to me to be entirely suitable for what you are doing and wanting to do so it's just a question of how to do it to avoid wearing the SD Card out.
 

hippy

Technical Support
Staff member
Ibenson & Hippy,
This is a typo. Try searching for "Raspberry Pi Pico"
Yes, I am assuming it's 'a typo' but I cannot see that julianE was meaning "Raspberry Pi Pico" as that is not a device which runs Linux nor uses SD Card.

A link from julianE to whatever they have, what this "Nano" is, will be the easiest way to identify what it actually is.
 

julianE

Senior Member
I am terribly sorry, it's raspberry pi zero W. I have no idea why I called it nano, dementia knocking on my door.
Thanks all for the help. Flesner, I never knew about the wear leveling, excellent information, I will be studying your links. I have been using the cheapest SD cards i can find, Microcenter brand, often under $5 for a card. It's perfectly fine for my use but I'm setting up a data collection system for my sister and I would like it to last.

I think I pieced it together why I called it nano, raspberry pi has a new product pico so my brain made a weird connection, I just bought a pico to give it a try.

Thank you all for all the help, best forum on the internet and great to have Hippy back with us.

Happy New Year to all and let's hope if there are not more of us that there are not less.
 

hippy

Technical Support
Staff member
it's raspberry pi zero W.
Many thanks for the clarification. As suspected that should be entirely suitable for what you need and I would concentrate on making that work rather than heading in some other direction.

I still believe wearing out SD Cards is an over-stated problem, that there's far more evidence they last for years under typical use than they don't. I have also used the cheapest SD Cards I can find and have not had a single card fail with my use.

A lot about "SD Cards will wear out" reminds me of "batteries suffer memory effect". Not entirely myth but an over-extrapolation of a rather specific and rare phenomena.
 

lbenson

Senior Member
My solution to the SD card failure problem (which was real for me), was to fit one of the 18650 backup modules with USB out, and supply the pi from that. But the modules that I used had a fault--if the power went off and the battery ran down, it wouldn't restart once power was restored because the draw from the pi meant that the current left over for charging was below a threshold that the module had set. My solution for that was to run a wire from the micro USB input 5V directly to the USB output 5V (with Schottky diodes between the wire and the output and inline with the module's feed to the USB 5V (after cutting a trace)).

I hope at some point they will fix this obvious failure mode so that no hacking is required (but how will I know when that is done?).
 

julianE

Senior Member
18650 backup modules with USB out, and supply the pi from that.
I do hope to run a pi at a remote location from a battery with solar charging so I will note your suggestions. My plan is to have a picaxe with rf module power the pi on for when i need it. Plan is to have a picaxe collecting data. Pi will be off charging the rest of the time. the beautiful part, I can program the picaxe using pi. It will be a multiple picaxe system with one picaxe just powering the pi on and off. I actually started the work on it pre corona have to dig it up and work on it some more.
 

hippy

Technical Support
Staff member
My solution to the SD card failure problem (which was real for me), was to fit one of the 18650 backup modules with USB out, and supply the pi from that
This sounds like a typical attempt to avoid SD Card corruption which can occur when the power is suddenly pulled. That is a very real phenomena but entirely different to cards wearing out.
 

Flenser

Senior Member
Yes, I am assuming it's 'a typo' but I cannot see that julianE was meaning "Raspberry Pi Pico" as that is not a device which runs Linux nor uses SD Card.
My apologies for any confusion I created, I was so sure julianE's "Nano" typo meant the new Pico.
I should have known that it couldn't have been the Pico which, as Hippy pointed out, has no SD card slot.
 

papaof2

Senior Member
The RPi 3 I've been dabbling with for several years has "eaten" its SD card a couple of times and it's protected by a UPS, so not a power problem. First time, a reflash of the card worked. Next time it didn't. I did no diagnostics, just replaced the card.
The only purpose of the RPi is is to satisfy my curiosity ;-) Real work still mostly falls to the PICAXEs.
 

julianE

Senior Member
The PI that had a failed SD card now has a high quality SD card installed, we'll see how it fares. Incidentally, one of my early PIs lost an SD card, it's been running continuously for at least 6 years, no camera and all the pi did was receive barometric pressure from an arduino through a usb port and update thingspeak every 5 minutes or so. ran on a very early pi OS, i'm thinking the simplicity allowed the SD card to last.
 
Top