Depends on your definition of "inexpensive" and if the car is intended to be daily driven or not.
If this is your primary car - then the performance numbers you are looking at + reliability will NOT be very cheap. Engine swap alone will be a major undertaking - if this is your first time, the initial costs (equipment, etc.) can be quite staggering. Fortunately, there are lots of options for the 7th gen Corolla - as all A-series engine will physically fit on the mounts, the choice becomes how much you want to invest in the engine and what your ultimate power goals are. Even the OEM 1.6L 4AFE and 1.8L 7AFE engines can put up some decent numbers on boost. With a decent tune on the stock engine running 6-7PSI - you could see a realistic 50-65WHP gain.
Now what about costs? In the above example, assuming that the engine is in good enough shape to take boost and that the setup is properly tuned up (couple of dyno pulls, tuning of EMs system, etc. - you are looking at $3K-$5K for that kind of gain. Sounds like a lot of money, and it is, but you are also bumping up the power of the original engine by around 50% or so - which is a huge gain. We didn't even take about brake upgrades, exhaust and intake considerations, suspension upgrades, and transaxle considerations (beefier clutch, pressure plate, flywheel on manuals - upgraded valvebody and aux cooling for automatics).
To shoot for an engine swap and then turbocharge it, along with all the chassis upgrades and proper tuning so it doesn't blow up on you - could be easily looking tat $5K-$7K to start. Not to mention all the increased maintenance costs (synthetic fluids to take the higher heat, more aggressive fluid change interval, premium gas and thirst for more gasoline, etc.) Shooting for a 200-250WHP project car - that is attainable, but you need some serious amount of cash to get there. Not trying to discourage you - but you have to take in all the costs initially, during the build, and future maintenance. I've had other project cars in the past that had similar power gains (was a Honda guy in the early 90's, before the import craze took off - when we had to fabricate almost every single part for swaps and forced induction), but the amount of time and energy spent on them - I could have saved up that money and bought the car I wanted, and still keep the original car as a daily driver or spare car.
That said - for the 200-250WHP power goal and keep decent reliability - you are looking at 4AGE or 4AGZE as possible starting points. The 4AGZE already came from the factory with a supercharger - most drop that and set the engine up for turbocharging. As all the important bits are already in place (lower engine compression ratio, oiling system to help cool the piston, etc.) Still in relatively high demand, so expect to pay a premium for them. The 4AGE gives you the most flexibility - be it 16 valve or 20 valve variants, big-port, small-port, TVIS system, etc. - do you want something for higher RPM power or something for forced induction. In naturally aspirated form - the Formula Atlantic variants (16-valve, no emissions, etc.) were pushing 240HP+ from 1.6L of displacement. So lots of potential there, hard part is finding ones still in good shape. Donor engines can be found running anywhere from $650 to as much as $2K - depending on condition and source. You also have to verify the legality of the swap in your area - some places will not allow you to legally swap an older engine into a newer car - most of the engines I've listed were in cars that may be older than your Corolla and may not even be available in the US. Have to have them imported, which raises emissions flags for some jurisdictions and additional costs to you.
