There is no one correct answer here. Each project needs to be evaluated and deliverables should be discussed with the client PRIOR to any work being started.
Print projects consisting of brochures, business cards, books, manuals, sales pieces, etc. generally require only a PDF deliverable. The native files used in construction the project are retained by the designer. There are some cases where the native files are desired by the client. This, again, should all be worked out prior to work starting and prior to any quotes being given.
Web projects consisting of web sites, banners, etc, generally require HTML/CSS text files in many cases. There are times when a client wants layered Photoshop files. And sometimes the client merely wants png files. There's no one answer here.
Logotypes/branding - Logos should always be delivered in a vector format in addition to any formats such as png or jpg.
Each project will have it's own needs and tool requirements so its difficult to give a single answer to your question.
I have no problem delivering a layered PSD file for a web page if that's what the client wants. Of course, I need to know this is what they want from the start because I don't design web sites with only Photoshop. I'll code pages and build images to drop into the code. It's exceptionally rare for me to build an entire page with Photoshop. It's just faster and more accurate to build code. I may prototype or sketch with Photoshop but that's generally as far as I go with that tool. I've run into too many cases where I here "The text in the design we approved doesn't match the text on the web site." And trying to explain the difference between image text and live HTML text can be impossible with some clients.
For print work I pretty much never release native files whether they are Photoshop, Illustrator, or Indesign files. Deliverables are customarily only a PDF. If the client wants the native files, I'll provide them for a fee. They are never free of charge.
For branding/logos I provide a native vector format, then additional formats as I see fit. This means you can't merely use Photoshop and be done. You should use Illustrator, CorelDraw, Freehand, Xara,... some vector application. Photoshop, while a great tool, never creates true vector files. Generally a logo delivery consists of .ai, .eps (vector), .pdf, .psd, .png, and .jpg. I also provide color breaks for each format. So that means an RGB .ai files, a CMYK .ai file, a Spot color .ai file, and so on. any raster-based files beyond a PSD are provided for client convenience. The .png or .jpg or even .gif are merely provided because clients expect to see those format because they are familiar with them. These formats also provide immediate use for many of the clients own needs (throw it on their facebook page, etc).