This video helps to solve Prius Gen 3 rattling & knocking noise. Local mechanic suggested to junk the vehicle because the problem might be a blown head gasket, which is not worth for a fix. I thought otherwise, and tried my luck to identify the problem on my own, which my luck shone on me. It was a simple fix. Changed 1 spark plug, changed 1 coil spring, and poured a bottle of Techron Concentrate Plus complete fuel system cleaner. Problem solved :). No more shaking and rattling the engine.

For those having similar issue, I'd suggest the following 3 things:
1. Check the spark plugs (change if needed) Spark plugs are cheap, and since you spent the time removing those, why not change them now.

2. Check the spark plug's coil springs (change if needed). Coils springs are cheap. I got mine for $15/piece. Find those on Amazon or local parts store.

3. Instead of cleaning and changing the fuel injector, for the sake of labor, I'd suggest to use above mentioned Fuel system cleaner. Pour 8 oz bottle while your tank is almost empty and filled it up. Drive 100 miles, it should clean your fuel system. I'd recommend using the same method at every oil change or 5000 miles.