Should only need to be hand tight - sometimes coating the threads with some oil or grease can help. But from your description, sounds like the compression test was fine.
Oil that is pooled down in the sparkplug wells will cause a misfire. Pretty simple matter to change out the valvecover gasket - will also give you access to the cams and valvetrain to see how everything is looking.
Checking coils - that is exactly how you check them. They make an a SST adapter cable to lengthen the wiring to the coil for easier testing, could be hard to find though. Simplified steps - do this only over a couple of cranks - otherwise you could load each cylinder with a lot of fuel - watch for accumulated gasoline vapor - you could disable fuse to fuel pump to prevent it from pumping any gas to start out with, though some gas will still be in the fuel rail. Stick plug into coil pack, ground plug to chassis ground, crank engine and observe spark.
Check for vacuum leaks - you could use carb cleaner or similar solvent, just have to watch what you spray and note that it will soften the rubber/plastic bits (ie, eat the part you just sprayed). Soapy water in the spray bottle can work just as well, in that manner. Just hit the hoses around the manifold, hose unions, vacuum hoses and check valves on the brake booster assembly, around throttle body, around intake manifold - that should be sufficient. There are other vacuum areas - EVAP for example, but those would not affect the idling and performance in the manner you are mentioning.
When you changed the O2 sensor - which one did you end up with? Denso OEM or aftermarket? Some 1ZZ-FE engines are quite picky, some don't seem to like anything other than Denso OEM - possible impedance mismatch issue. Same with the coils - did you get OEMs one or aftermarket? Probably not a bad idea and check all chassis grounds and wiring to the coils/injectors. Another area this engine is sensitive to is electrical noise. Too much noise can lead to unexpected behavior that can be really tough to diagnose.
Given that you replaced the plugs, O2 sensor (assuming upstream one), two middle coil packs, and verified compression and assuming the injectors, fuel delivery, and the induction system are fine - tt almost sounds like the engine is missing due to a ignition timing issue. Unfortunately, you cannot "slew" the timing on a modern car anymore, without the Toyota handheld scanner. All that is set by the ECM. You can still double check the timing on the car though - see if it advances it as RPMs climbs. VVTi system malfunction could also be a possibility - though diagnosing that is a little more involving. Quick check is the disconnect the OCV valve and see if the idle changes or not. If it doesn't change - then the VVTi/OCV system is suspect.