For typical images, a pixel is a single byte. This is the smallest atomic unit. It is a number from 0 to 255 and is interpreted as a grey value, 0 being black, 255 being white, and 254 values between. For a greyscale image, there is only one channel called grey, for RGB color, there are three bytes stored "per pixel": one value per color channel per pixel. (CMYK has 4 greyscale channels; any alpha channels adds another greyscale channel etc.)
For your 1024 x 1024 example, the smallest unit is a pixel without any regard to the ppi value, which is not relevant. The smallest editable block will always be a pixel. In your example, there are no inches. Therefore, Pixels per Inch cannot be calculated and has no meaning, neither philosophically nor existentially.
You may have seen people say that screens have 72ppi etc. This is a "lie for children." It is obvious that two 1024x768 screens, one which is 20 inches, one which is 13 inches will have a different ppi.
PPI is the same sort of number as MPH or KPH.
360 miles, 60mph means 360/60 = 6 hours
360 miles, 100mph means 360/100 = 3.6 hours
360 pixels, 60ppi means 360/60 = 6 inches
360 pixels, 100ppi means 360/100 = 3.6 inches
It is important to notice that, just like "60mph" tells you nothing about the time or the distance to any individual destination, 60ppi tells you nothing about the physical dimensions of any individual image.
Resolution has multiple definitions.
In the case of making images it simply means the number if pixels. When referring to the resolution of a screen or image, this is the typical definition. This is the actual pixel data. Altering this alters data. Given the huge variety in screen sizes and native resolutions, for screen and web, inches generally do not exist and so ppi is irrelevant. (It is probably better to say "inches are undefined.")
In the more general case, it means the number of physical units represented by an individual pixel. When referring to ppi, this is the meaning intended. For image manipulation and storage on disk, this is merely a tag. Altering this does not alter pixel data. (Notice in the pixel example above that the inches change without any alteration in the pixel count).
In many programs, the user interface will ASSUME you want to alter the pixel data when altering the ppi flag. I think this is a source of confusion for many people. In photoshop it can be turned off, but in the image resize dialog, if you change the ppi field, it will alter the pixel size automatically. If you reset the pixel dimensions back to what they were before hitting "ok" you will see that the file size estimate shows no change.
Note that if I set an image to some ginormous ppi value in photoshop, and set it to 3 inches in DTP software, the RIP software which makes the plate is probably going to resample it to something else based upon the line resolution of the CMYK halftone screens.
One final point is that if you keep the ppi (or MPH) fixed (say 100) and you want to a specific inches (time) factor (say 4), then the only number left to manipulate is the pixels (miles):
400 miles, 100mph means 400/100 = 4 hours
400 pixels, 100ppi means 400/100 = 4 inches
P.S. For printing, there is a rule of thumb that you want to provide 300ppi. All this means is that you want to provide 300 pixels for every inch in size you want it printed, the dpi flag means nothing only the number of pixels matters.
