Do you have a source for these batteries and their charger.?
Here is my latest for nearly-off-the-shelf 18650 battery holder + charger UPS:
Like this:
https://www.aliexpress.com/item/4000200690185.html
As this youtube video points out, there's an issue with these if used as a ups--if mains gives out and the battery is discharged, if the load upon the return of power exceeds a certain, small, amount, the battery won't recharge:
I've made the following modifications to get around this problem:
As shown in the photo, I cut the trace between the ON switch pin and the V+ pin on the usb connector and replaced that link with a 1N5818 schottky diode. I ran a wire from the diode near the USB charging connector (S1) to another 1N5818 also connected to the same V+ pin. If mains comes back on when the battery has been discharged, the mains current supplies the load with (assuming not too great a load current) a surplus to charge the battery.
This has not been extensively tested, but seems to work--if your circuit can tolerate a little less than 5V because of the schottky diode voltage drop.
These are available with slots for 1, 2, or 4 18650 batteries. Runtime on battery will depend on how good your batteries are. I always use "protected" 18650 batteries because of various dangers. These battery chargers are also quoted as having battery protection. Belts and suspenders, I hope.