So, despite having putting together some of the projects from here, I'm still lacking in the technical knowledge of a lot of this stuff. I'm definitely still a newbie to DIY electronics in general, haha

When you say as long as it isn't below 3V, does the V decrease over time, or when you leave it sitting? Or is that like what the battery just normally pumps out and has nothing to do with capacity? The only thing I have to test it is a little board from adafruit that lights up blue when a battery is plugged into it; it's an Adafruit Powerboost1000 Charger ( https://www.adafruit.com/product/2465 ). I plugged the battery in an it lit up so I just automatically thought "Ah it's still working!", but of course I'm not really sure.
I know I can use a multimeter for a proper test. I bought a multimeter from Amazon that broke on me right out of the box, so I never really learned how to use it. I actually bought it thinking that I would be using it all the time, but despite that, I never had a chance to, so I let it sit for awhile, then I decided I wanted to use it for funsies and found that it didn't work. I was way passed the time of return window.
Thanks for the reply by the way! Learning about this stuff is a ton of fun!
