The feed to a switch loop used to be given special dispensation in the code from having to be remarked. In the current code you are (and you should in your work) remark it.

As you've shown it (assuming you are using the same devices), your circuit will not work. The switched half of the outlet has no power feeding it ever and neither does the switch and the resultant lights.

If you don't want to retained the outlet as switched, you can feed the switch (and the downstream lights) with the hot in the receptacle box (you'll need to replace the receptacle or jumper the "split" connection on it).

I was planning on putting in a new receptacle since the brass tab connecting the two plugs is currently broken off. If I put in a new receptacle, does my proposed wiring plan look correct?

Thanks,
Eric

Yes, and while technically acceptable, I do not like making through connections on the device. I'd connect the black feeding to the switch to the others under the wire nut instead. The rest is fine (presuming that the omitted grounds on all the cables, devices, and any metal boxes are indeed connected together).

You can feed the new lights from the existing switched receptacle feeds or form the switch itself (if the switch is not on a switch loop).

The choice may depend on joist direction and light placement.

Running wire parallel to the joists allows it be be fished above the ceiling, running perpendicular to the joists means you need to bore a hole in every joist for the cable.
The holes at every joist require patching (and if the cable is not 1.25 inches from the bottom edge of the joist a nail plate to protect the cable).

Since you recommend not running the connections to the switch and next outlet through the receptacle, should I also have the white wire that goes to the next outlet in the white wire nut bundle just like you recommended for the black wire?

Since I have a walk up attic I was planning on running the wires up to the attic and then drill holes 3/4" holes through the middle of the joists to run the wires. I have 2x10 joists so I think I am all set just running the wires through the holes I will drill in the centers.