For example, a JPG cannot hold transparency; it must have a background color of some type. However, as a tradeoff, it is what is called a "lossy" format, which means that some data will be lost when it is saved due to being compressed.
PNG, however, is saved in a "lossless" format (as-is), which means the file you see as a PNG is eye-to-eye identical to what was originally saved (in many cases, a PSD file). PNGs can be slightly compressed, but not by a lot.
JPGs vary in quality, from very bad to extremely detailed, and each has a consequential filesize. In general, a low-quality JPG will have a small filesize, and a high-quality one will have a filesize. Surprise, surprise...
Here's an example of what I mean, saving the same PSD of the text "Potato" on a gradient background in six different formats.
The highest quality belongs of course to image 4, the lossless export of the image in PNG format. However, image 6, the web version (which is ~2kB smaller), is almost identical. When layered one on top of the other in PhotoShop, and applying the "Difference" blending tool, you simply get a black image--which means there is literally no pixel-by-pixel difference. (At least for our test case.)
Now let's look at the JPGs--images 1, 2, 3, and 5. Image 3, the highest-quality JPG possible, still sports some TINY JPG artifacts around the lettering, though it's hardly visible. Image 2, half-quality, shows some far more visible artifacts and image 1, the lowest, is just garbage.
If we look at image 5, though, the web export (JPG High) from PhotoShop, its filesize is less than half of the PNG web export. This is a big deal for a lot of people, and thus they use it in place of PNGs to protect bandwidth. While this is a huge problem for companies like Google, who experience millions of hits a day, it's not so much for small-time blogs, even one like Gizmodo.
Just to be clear on that "small" difference, here is our logo in: A) PNG format, B) PNG format with black level bumped up about 80%, C) JPG format, D) JPG format with the same transformation. The level change just helps to show where the JPG artifacts are hiding.
Depending on the monitor (and settings), attention to detail, and browser you have, you may or may not notice the JPG artifacts haloing your text--but beware... someone might.