Compact Date Storage & Retrieval

cpedw

Senior Member
I am trying to make a Calendar/Appointments device. Clearly, date and time storage are significant. TIme of day can readily be stored in a byte at 10 minute intervals. Date is more involved. I found this post describing a modified Julian Date Number (JDN) calculation which covers the years 2000-2178 in one word.

So far so good. My problem is to restore Day, Month and Year from the modified JDN. Wikipedia describes a method to restore Gregorian dates (I hope that's the correct one!) but it's very complex and I can't get it to work on a spreadsheet, never mind making it Picaxeable.

Has anyone tackled this problem with a Picaxe before? Is there an alternative approach that I should consider? Or must I persevere with the Wikipedia method until it yields?

Derek
 

AllyCat

Senior Member
Hi,

Note the first line from the thread that you linked above: "Here is a program for calculating Julian day numbers with a base date of 01-Jan-2000 (= day zero) ." Both (Modified) Julian and Gregorian calendars use much earlier starting dates so have much larger day numbers. It appears that the MJD is due to overflow a 16 bit number moderately soon, so doesn't appear to be a sensible choice (nor the Gregorian Date number).

Personally, I would just work from a suitable reference date such as 1st January 2000 or 2020 and then use an offset value (of days) if a "real" Julian or Gregorian Date is ever encountered. That's what I did in my (not directly relevant) Sunrise and Sunset calculator which surfaced again recently.

For the "reverse" calculation (i.e. day number to DMY) you can just divide by 365 (to indicate the Year number) and use the PICaxe // operator to find the "remainder" number of (days). Then introduce an "offset" to take into account the number of leap years between the two dates, basically dividing the number of years by 4 (at least until the year 2100). Decide whether your reference January 1st is "Day Zero" or "Day One" and use some Trial and Error if necessary, particularly around the December-January and February 28th dates.

IMHO you are much more likely to encounter practical "issues" with Daylight Saving Time and/or Time Zones in general, for which you need a precise strategy, depending on how the data will actually be used. ;)

Cheers, Alan.
 

oracacle

Senior Member
I agree with AllyCat, but once you know where you are in the year it can be done with simple if statments.
this code was used with DS3231, whcih showed sunday as 1, then check for 1am and change the time if that was the case
Code:
    if month = 3 and date > 24 and day = 1 and hour = 1 then
        hour = hour + 1
    end if 

    if month = 10 and date > 24 and day = 1 and hour = 2 then
        hour = hour - 1
    end if 

    end
Leap years are easy, does it divide by 4, but not 100? but if it divisible by 100, is it divisable by 400, if so it is not a leap year. I knoiw, but i don't make the rules. its all maths that a picaxe can do
How to Calculate Leap Years: 7 Steps (with Pictures) - wikiHow

just store the number of days from a set date, like 01/01/2020, divide 365 to get the year. The tricky part is the maths behind figuring out if you should be adding in the leap days, and how many you should add, 1 day for each year acording to to the above informtaion. This is similar to how unix time works, only that has been counting seconds since 1st january 1970 and computer works the time out from there. It is due to overflow in 2038 as its only a 32bit interger. Its another situation that we know is going to happen but nobody seems particularly bothered about getting sorted out. Solve your 178 year problem and we might use it to solve the unix time problem
 

papaof2

Senior Member
For a date reference used internally in a program handling only future dates, it doesn't matter what your base year is. 2020 is an even number and probably a good starting place. 2070 is 50 years out and well within 16 bits. Will your device/code still be in use in 50 years?

I have some hand tools which are older than that but there's not much electronics older than 30 years - other than a console AM/FM radio with phonograph and cassette recorder/player (at least the radio still works ;-), a couple of CB radios and some ham radio gear.
 

rq3

Senior Member
I have a 48 year old HP-35 calculator that still works! As does my HP-67, after replacing its mag card drive rollers. But my 63 year old mechanical Curta "math grenade" is my go-to calculator, and the batteries have never worn out.
 

hippy

Technical Support
Staff member
I am not convinced it is worthwhile doing 'clever maths' when bit fields will cater for 127 years.

Code:
#Picaxe 20X2

; Date : 54321-9876543210
;        yyyyyyymmmmddddd

#Macro MakeDateStamp(wDateStamp, wYear, bMonth, bDay)
  wDateStamp = wYear - 2020 << 4 | bMonth << 5 | bDay 
#EndMacro

#Macro ExtractDate(wDateStamp, wYear, bMonth, bDay)
  wYear  = wDateStamp >> 9 + 2020
  bMonth = wDateStamp >> 5 & 15
  bDay   = wDateStamp      & 31
#EndMacro

; Time : 54321-9876543210
;        hhhhhmmmmmmsssss

#Macro MakeTimeStamp(wTimeStamp, bHour, bMins, bSecs)
  wTimeStamp = bMins <<  6 | bSecs >> 1
  wTimeStamp = bHour << 11 | wTimeStamp
#EndMacro

#Macro ExtractTime(wTimeStamp, bHour, bMins, bSecs)
  bHour = wTimeStamp >> 11
  bMins = wTimeStamp >>  5 & 63
  bSecs = wTimeStamp <<  1 & 63 
#EndMacro

#Macro DetermineDayOfYear(wDayOfYear, wYear, bMonth, bDay)
  ; Day of 1st    Jan Fb Mr Ap May Jun Jly Aug Sep Oct Nov Dec
  LookUp bMonth, (0,1,32,60,91,121,152,182,213,244,274,305,335), wDayOfYear
  ; Adjust for how far into the month we are
  wDayOfYear = wDayOfYear + bDay - 1
  ; Adjust for leap year
  w0 = wYear % 4
  if w0 = 0 Then        ; Divisible by 4 so could be a leap year
    w0 = wYear % 400
    If w0 = 0 Then      ; And is if divisible by 400
      wDayOfYear = wDayOfYear + 1
    Else
      w0 = wYear % 100
      If w0 <> 0 Then   ; But only if not divisible by 100
        wDayOfYear = wDayOfYear + 1
      End If
    EndIf
  EndIf
#Endmacro

#Macro DetermineDaysInYear(wDaysInYear, wYear)
  DetermineDayOfYear(wDaysInYear, wYear, 12, 31)
#EndMacro

Symbol reserveW0 = w0  ; b1:b0

Symbol todayDate = w1  ; b3:b2      yyyyyyymmmmddddd
Symbol todayTime = w2  ; b5:b4      hhhhhmmmmmmsssss

Symbol year      = w3  ; b7:b6      2020-2147
Symbol month     = b8  ;            1-12
Symbol day       = b9  ;            1-31

Symbol hour      = b10  ;           0-23
Symbol mins      = b11  ;           0-59
Symbol secs      = b12  ;           0-59

Symbol doy       = w7   ; b15:b14   1-366    Day of year
Symbol diy       = w8   ; b17:b16   365-366  Days in year

MakeDateStamp(todayDate, 2021, 10, 25)
MakeTimeStamp(todayTime, 14, 43, 57)

ExtractDate(todayDate, year, month, day)
ExtractTime(todayTime, hour, mins, secs)

DetermineDayOfYear(doy, year, month, day)
DetermineDaysInYear(diy, year)

SerTxd(#year, "-", #month, "-", #day,  " ")
SerTxd(#hour, ":", #mins,  ":", #secs, ", ")
SerTxd("Day ", #doy, " of ", #diy, CR, LF)

Do While year > 1970
  year = year - 1
  DetermineDaysInYear(diy, year)
  doy = doy + diy
Loop

SerTxd("Day ", #doy, " since Jan 1st 1970", CR, LF, CR, LF)

DetermineDaysInYear(diy, 2020)
SerTxd("Days in 2020 = ", #diy, CR, LF)
DetermineDaysInYear(diy, 2010)
SerTxd("Days in 2010 = ", #diy, CR, LF)
DetermineDaysInYear(diy, 2000)
SerTxd("Days in 2000 = ", #diy, CR, LF)
Code:
2021-10-25 14:43:56, Day 298 of 365
Day 18926 since Jan 1st 1970

Days in 2020 = 366
Days in 2010 = 365
Days in 2000 = 366
 

lbenson

Senior Member
#Macro MakeDateStamp(wDateStamp, wYear, bMonth, bDay)
. . .
#Macro MakeTimeStamp(wTimeStamp, bHour, bMins, bSecs)
Ah, it had been too long since I added to my collection of nifty hippy code snippets and macros.
 

cpedw

Senior Member
Curses. Having (I hope) overcome the clever maths hurdle, the simulator is currently checking that all 127 years are working correctly. It involves mainly simpe arithmetic but also 2 invocations per day of Alleycat's 32 bit divided by 16 bit routine which is quite slow, especially in the simulator. If it concludes satisfactorily, I will report back for completeness, In case anyone wants to work with Julian Date Numbers. Don't hold your breath.
Meanwhile, I will proceed using hippy's straightforward, elegant solution. Thank you.
 

cpedw

Senior Member
I have proved to my satisfaction that this subroutine can turn a Modified Julian Date Number, as generated by T Ikeda's program, back into Dat, Month and Year data. Compared to hippy's method above for date storage, it's slow and complex but in case anyone needs to work with JDNs, here it is. For completeness, I have included Alleycat's double word division routine here.
Code:
GenYMD:
'Date calculator from Julian day number (JDN)

'Input JDN Modified JDN, base 2000/1/1=0 (word)
Symbol JDN = w4    ' Modifed Julian Date Number based on 2000/01/01 = 0
'Output 
'    YYY - Year 0-178
'     MM - Month 1-12
'     DD -  Day  1-31
Symbol YYY = b10    '0 to 178, corresponding to years 2000-2178 input (max 2099 Reverser)
Symbol MM = b11    'Months: 1-12
Symbol DD = b12    'Day: 1-31

' Based on https://en.wikipedia.org/wiki/Julian_day#Converting_Julian_calendar_date_to_Julian_Day_Number but cut down to cover dates from 2000-2099 inc.
' DW 25/10/21

'Uses Alleycat's double word division routine https://picaxeforum.co.uk/threads/a-simple-double-word-division-subroutine-maximum-31-bits-by-15-bits.21494/
'This occupies 2 bytes & 2 Words, 1 of which MUST be w1.

'Variables used by the double word division routine
symbol tempb = b1            ; Temporary (local) byte
symbol numlo = w1            ; Must be w1 (to use bit flags), also used for result word
symbol numhi = w2            ; Or any other word, also used for remainder word

symbol divis = 1461

IF JDN>36583 THEN    'Adjustment for year 2100 not a leap year.
    JDN = JDN + 1
ENDIF
numhi = JDN + 306 ** 4
numlo = JDN + 306  * 4
GOSUB div31        'Divide divis into (numhi:numlo), Result numlo, Remainder numhi.
DD = numhi / 4 * 5 + 2 // 153 / 5 + 1
MM = numhi / 4 * 5 + 2 / 153 + 2 // 12 + 1
numhi = JDN ** 4
numlo = JDN  * 4
GOSUB div31        'Divide divis into (numhi:numlo), Result numlo, Remainder numhi.
YYY = numlo

IF JDN>36584 THEN    'Re-adjust to restore JDN
    JDN = JDN - 1
ENDIF
RETURN

div31:        ; Divide numerator (w2:w1) by divisor divis  - constant
   for b1 =  0 to 15                  ; Repeat for each bit position
   numhi = numhi + numhi + bit31        ; Start to shift numerator left (top bit lost)
   numlo = numlo + numlo                ; Shift low word
   if numhi >= divis then                ; Skip if can't subtract
       numhi = numhi - divis            ; Subtract divisor, then.. 
       inc numlo                      ; Add the flag into result (in low word)
   endif      
   next b1                    ; Typically 26 bytes, execution time 70ms
   return        ; Result in numlo (w1), remainder in numhi (w2), divisor (w3) unchanged
 

hippy

Technical Support
Staff member
I have proved to my satisfaction that this subroutine can turn a Modified Julian Date Number, as generated by T Ikeda's program, back into Dat, Month and Year data. Compared to hippy's method above for date storage, it's slow and complex but in case anyone needs to work with JDNs, here it is.
I am sure there must be an easier way to do it ... but I have no idea how to.

But here's T Ikeda's program converted into a Macro which makes it easier to install into a program, which is as far as I got ...
Code:
#Define EPOCH 2000

#Macro EncodeJulianDate(wJdn, wYear, bMonth, bDay)
  ; Algorithm from T.Ikeda - KA1OS - https://picaxeforum.co.uk/threads/8674
  wJdn = bMonth / 3 Max 1
  wJdn = wYear  - EPOCH + wJdn  * 7 / 4
  wJdn = wYear  - EPOCH * 367   - wJdn
  wJdn = bMonth * 3912  / 128   + bDay - 31 + wJdn
  wJdn = wJdn   / 36585 * $FFFF + wJdn
#EndMacro

EncodeJulianDate(w0, 2021, 10, 26)

SerTxd("With 'Year ", #EPOCH, "' as the Epoch - The Julian Day is ", #w0, CR, LF)
 
Last edited:

hippy

Technical Support
Staff member
Here's a different approach to dividing by 365.25 or whatever the traditional decoding algorithms require which leads to the maths being greater than 16-bit.

It works on the principle, when the epoch is a leap year, of there being four year cycles of 1461 days; a leap year then three non-leap years. So it's pretty easy to determine the year and how many days into the year we are.

If only I could find a day number to month plus day conversion algorithm we wouldn't need the iterative approach I use here.

Code:
Symbol EPOCH          = 2020  ; Must be a leap year

; Calculate JDN for 2100-03-01 as ((2100 - EPOCH) / 4 * 1461) + 59

Symbol K1             = 2100 - EPOCH
Symbol K2             = K1 / 4
Symbol K3             = K2 * 1461
Symbol JDN_2100_03_01 = K3 + 59

#Macro EncodeJulianDate(wJdn, wYear, bMonth, bDay)
  ; Algorithm from T.Ikeda - KA1OS - https://picaxeforum.co.uk/threads/8674
  wJdn = bMonth / 3 Max 1
  wJdn = wYear  - EPOCH + wJdn  * 7 / 4
  wJdn = wYear  - EPOCH * 367   - wJdn
  wJdn = bMonth * 3912  / 128   + bDay - 31 + wJdn
  wJdn = wJdn   / JDN_2100_03_01 * $FFFF + wJdn
#EndMacro

#Macro DecodeJulianDate(wJdn, wYear, bMonth, bDay)
  w0     = wJdn / JDN_2100_03_01 Max 1 + wJdn
  bMonth = 13
  wYear  = w0 / 1461 * 4 + EPOCH
  w0     = w0 % 1461
  ; At this point w0 is how many days into a four year cycle with a leap
  ; year at its start. So 0-365 is the first year, a leap year.
  If w0 <= 365 Then
    w0 = w0 + 1
    ; At this point wYear is correct, and is a leap year.
    ; And w0 holds the day of year 1-366
    Do
      bMonth = bMonth - 1
      ; Day of 1st    Jan Fb Mr Ap May Jun Jly Aug Sep Oct Nov Dec
      LookUp bMonth, (0,1,32,61,92,122,153,183,214,245,275,306,336), w1
    Loop Until w0 >= w1
  Else
    wYear = w0 - 1 / 365 + wYear
    w0    = w0 - 1 % 365 + 1
    ; At this point wYear is correct, not a leap year.
    ; And w0 holds the day of year 1-365
    Do
      bMonth = bMonth - 1
      ; Day of 1st    Jan Fb Mr Ap May Jun Jly Aug Sep Oct Nov Dec
      LookUp bMonth, (0,1,32,60,91,121,152,182,213,244,274,305,335), w1
    Loop Until w0 >= w1
  End If
  bDay = w0 - w1 + 1
#EndMacro

Symbol reserveW0  = w0 ; b1:b0
Symbol reserveW1  = w1 ; b3:b2

Symbol julianDate = w2 ; b5:b4
Symbol year       = w3 ; b7:b6
Symbol month      = b8
Symbol day        = b9

#Macro Test(wYear, bMonth, bDay)
  EncodeJulianDate(julianDate, wYear, bMonth, bDay)
  SerTxd(#wYear, "-", #bMonth, "-", #bDay, " -> ", #julianDate, " -> ")
  DecodeJulianDate(julianDate, year, month, day)
  SerTxd( #year, "-", #month, "-", #day, CR, LF)
#EndMacro

DecodeJulianDate(0, year, month, day)
SerTxd( "Earliest : ", #year, "-", #month, "-", #day, CR, LF)
DecodeJulianDate($FFFE, year, month, day)
SerTxd( "Latest   : ", #year, "-", #month, "-", #day, CR, LF, CR, LF)

Test(2021, 10, 27)

; 2020 was a leap year
SerTxd(CR, LF)
Test(2020,  2, 28)
Test(2020,  2, 29)
Test(2020,  3,  1)

; 2100 is not a leap year
SerTxd(CR, LF)
Test(2100,  2, 28)
Test(2100,  3,  1)
Code:
Earliest : 2020-1-1
Latest   : 2199-6-5

2021-10-27 -> 665 -> 2021-10-27

2020-2-28 -> 58 -> 2020-2-28
2020-2-29 -> 59 -> 2020-2-29
2020-3-1 -> 60 -> 2020-3-1

2100-2-28 -> 29278 -> 2100-2-28
2100-3-1 -> 29279 -> 2100-3-1
 
Last edited:

hippy

Technical Support
Staff member
If only I could find a day number to month plus day conversion algorithm we wouldn't need the iterative approach I use here.
And this seems to be that ...
numhi = JDN + 306 ** 4
numlo = JDN + 306 * 4
GOSUB div31 'Divide divis into (numhi:numlo), Result numlo, Remainder numhi.
DD = numhi / 4 * 5 + 2 // 153 / 5 + 1
MM = numhi / 4 * 5 + 2 / 153 + 2 // 12 + 1
Because this also repeats in a 1461 day cycle one can modify that as below and it still works ...
Code:
numhi = JDN // 1461 + 306 ** 4
numlo = JDN // 1461 + 306 * 4
GOSUB div31
And, with JDN limited to 0-1460, multiplying by 4 doesn't exceed 16-bit, and we can get the remainder with -
Code:
numhi = JDN // 1461 + 306 * 4 // 1461
DD = numhi / 4 * 5 + 2 // 153 / 5 + 1
MM = numhi / 4 * 5 + 2 / 153 + 2 // 12 + 1
Which means this works ...
Code:
#Macro DecodeJulianDate(wJdn, wYear, bMonth, bDay)
  w0     = wJdn / JDN_2100_03_01 Max 1 + wJdn
  wYear  = w0 / 1461 * 4 + EPOCH
  ... bug fix for wYear needed here ... included in code below
  w0     = w0 // 1461 + 306 * 4 // 1461 / 4 * 5 + 2
  bDay   = w0 // 153 / 5 + 1
  bMonth = w0 /  153 + 2 // 12 + 1
#EndMacro
So many thanks for that. and my full test code is ...
Code:
#Picaxe 20X2
#Terminal 9600
#No_Table
#No_Data

Pause 2000

Symbol EPOCH          = 2020  ; Must be a leap year

; Calculate JDN for 2100-03-01 as ((2100 - EPOCH) / 4 * 1461) + 59

Symbol K1             = 2100 - EPOCH
Symbol K2             = K1 / 4
Symbol K3             = K2 * 1461
Symbol JDN_2100_03_01 = K3 + 59

#Macro EncodeJulianDate(wJdn, wYear, bMonth, bDay)
  ; Algorithm from T.Ikeda - KA1OS - https://picaxeforum.co.uk/threads/8674
  wJdn = bMonth / 3 Max 1
  wJdn = wYear  - EPOCH + wJdn * 7 / 4
  wJdn = wYear  - EPOCH * 367  - wJdn
  wJdn = bMonth * 3912  / 128  + bDay - 31 + wJdn
  wJdn = wJdn   / JDN_2100_03_01 Max 1 * $FFFF + wJdn
#EndMacro

#Macro DecodeJulianDate(wJdn, wYear, bMonth, bDay)
  ; Algorithm from cpedw - https://picaxeforum.co.uk/threads/32492
  w0     = wJdn / JDN_2100_03_01 Max 1 + wJdn
  wYear  = w0 /  1461 * 4 + EPOCH
  w0     = w0 // 1461 * 4
  wYear  = w0 /  1461 + wYear
  w0     = w0 +  1224 // 1461 / 4 * 5 + 2
  bMonth = w0 /  153  + 2 // 12 + 1
  bDay   = w0 // 153  / 5 + 1
#EndMacro

Symbol reserveW0  = w0 ; b1:b0

Symbol julianDate = w1 ; b3:b2
Symbol year       = w2 ; b5:b4
Symbol month      = b6
Symbol day        = b7

#Macro Test(wYear, bMonth, bDay)
  EncodeJulianDate(julianDate, wYear, bMonth, bDay)
  SerTxd(#wYear, "-", #bMonth, "-", #bDay, " -> ", #julianDate, " -> ")
  DecodeJulianDate(julianDate, year, month, day)
  SerTxd(#year, "-", #month, "-", #day, CR, LF)
#EndMacro

DecodeJulianDate(0, year, month, day)
SerTxd( "Earliest : ", #year, "-", #month, "-", #day, CR, LF)
DecodeJulianDate($FFFE, year, month, day)
SerTxd( "Latest   : ", #year, "-", #month, "-", #day, CR, LF, CR, LF)

Test(2021, 10, 28)

; 2020 was a leap year
SerTxd(CR, LF)
Test(2020,  2, 28)
Test(2020,  2, 29)
Test(2020,  3,  1)

SerTxd(CR, LF)
; 2100 is not a leap year
Test(2100,  2, 28)
Test(2100,  3,  1)
Code:
Earliest : 2020-1-1
Latest   : 2196-6-5

2021-10-28 -> 666 -> 2020-10-28

2020-2-28 -> 58 -> 2020-2-28
2020-2-29 -> 59 -> 2020-2-29
2020-3-1 -> 60 -> 2020-3-1

2100-2-28 -> 29278 -> 2100-2-28
2100-3-1 -> 29279 -> 2100-3-1
Note : I seem to have fallen into the habit of using "%" for modulo rather than "//" which is usually used with PICAXE ! Both are exact equivalents. I have updated this post to use "//".

Edit : Bug fix for wYear when decoding added.
 
Last edited:

cpedw

Senior Member
Excellent stuff ... but I think there's a gotcha in the DecodeJulianDate macro. The year calculation should be
Code:
wYear  = w0 * 4 /  1461 + EPOCH
 

hippy

Technical Support
Staff member
I believe my code is correct, determines the year of the start of the four year cycle. What I forgot to do was to add 1, 2 or 3 to that based on how far through those 1461 days we are. That's in the iterative code in post #13 but disappeared when I block deleted. Thanks for noticing.

While your code is mathematically correct it overflows 16-bit so I am currently sticking with ...
Code:
  wYear = w0 / 1461 * 4 + EPOCH
  w0 = w0 // 1461
  If w0 > 365 Then
    wYear = w0 - 1 / 365 + wYear
  End If
Edit : No, I'm going to use your "* 4 / 1461" trick ...
Code:
#Macro DecodeJulianDate(wJdn, wYear, bMonth, bDay)
  ; Algorithm from cpedw - https://picaxeforum.co.uk/threads/32492
  w0     = wJdn / JDN_2100_03_01 Max 1 + wJdn
  wYear  = w0 /  1461 * 4 + EPOCH
  w0     = w0 // 1461 * 4
  wYear  = w0 /  1461 + wYear
  w0     = w0 +  1224 // 1461 / 4 * 5 + 2
  bMonth = w0 /  153  + 2 // 12 + 1
  bDay   = w0 // 153  / 5 + 1
#EndMacro
I would have spotted the bug if I had used more exhaustive test cases - Test(2023, 12, 31) - Had been paying closer attention to what it was printing for year value. I got overly focused on 2100 and leap year handling.
 
Last edited:

cpedw

Senior Member
That all tests out OK.

A useful feature of the Test macro is that it can be used to validate a date. Invalid dates will generate a JDN but Decoding the JDN returns a different Y/M/D - quite handy.
 
Top