Hi,
The problem with a "floating" input is that the PICaxe's behaviour is "unpredictable". If the pin happens to read as "Low" (less than about 1.5 volts relative to Ground) then the PICaxe will start-up completely normally. This is why there are often posts from members saying that their program had worked perfectly and then suddenly stopped working for no apparent reason. Conversely, AFAIK if the pin remains "High" then the PICaxe will NEVER start, because it is waiting to receive a new Program download. It appears that in your case, the pin is starting High and then falls to Low after some seconds, which actually is not particularly surprising:
Any PICaxe I/O pin has a "stray" capacitance of some picoFarads to Ground and also to at least its adjacent pins. Leg 1 is the supply rail so when power is applied to the PICaxe, it is quite possible that stray capacitance or resistance will also pull Leg 2 (the serial Input) towards the supply voltage. Then the PCB tracks and/or any connected cables can add more stray couplings (Capacitive and/or "Resistive"), for example from the Serial Output and Ground connections.
The problem is that to change the voltage across a few pF in a few seconds needs only PicoAmps (i.e. millionths of a uA), which can flow in a "resistance" of millions of MegOhms. There also may be thermal leakage currents within the PICaxe chip itself, and furthermore when the PICaxe starts up it may drive the serial Output pin Low and/or apply a "handshake" signal to indicate that it is waiting for a new Program download.
Hence it is essential that the Serial Programming pin is normally held Low (usually a "High" can be used as a "Hardware Reset" input if required), but even my 100k (to ground) is quite conservative, it's probable that 1M ohms or even 10M or 100M may be sufficient in most applications.
Cheers, Alan