I'm a beginner in Illustrator and I’m trying to just fill in between the lines but it spilled over as shown in first pic. When I used shape builder to delete the spill it makes this weird line as shown in 2nd picture.
The path should be closed if you want the fill to be contained. It makes sense too. You can't contain something if it's open.
Looks like you've tried Fill in one of those two curved lines and it is open. Try joining the ends of both curves so it creates one closed shape.
What the shape builder does is basically creates 2 partitions, so the color is remained inside one of them. If you see both partitions are now closed unlike your original shape.
One way to join the ends of those curves is Cntl + J (on Windows):
Note that once you join, the appearance of curve widths will also change. If you expect it to be final version of your design, I suggest you select both shapes (without fill) and
Object > Expand them. And then use Shape Builder or Paint Bucket to fill color.