Although I have been using ArcGis as starting point in my GIS learning in last project I was using QGIS. I am using this opportunity to tell you that this FREE GIS packages is really great and in the most cases it can substitute ArcGis.
I do not want to tell you that ArcGis is bad, on the contrary, it is great package and there is a lot of educational material for it (including my 7 hours online course) but its problem is that it is not free.
So, if you want to dive into GIS waters for FREE, I am warmly recommend you QGIS which can be downloaded on its official site http://qgis.org/en/site/
If you have problems with starting to learn it feel free to contact me, I can show you basic in 30 minutes.