If they didn't like you, they would not have interviewed today - Job interviews are extremely time consuming and expensive and most employers will only bring back a second round candidate if they are genuinely interested. First round candidates are often there not make the numbers up (especially in public sector jobs), to be seen to be going with procedure, but not second round.
Having said that, the employers will want to know they have the best possible candidate, so don't be too disappointed if they do find someone else to interview too. Think of it as throwing a dice. If you get a six this time, you've got the job. If not, you gave it your best shot, move on, try again.
Even if they do behave sh*tty, remember this process is about you finding the right employer too; who wants a job with an employer that's already stabbing you in the back before you even get the job?! !!
Re: references, I don't know about where you're from, but here in the UK, most big employers only give very minimal references - "we can confirm W worked here for X years as Y". Anything else is a court case or tribunal waiting to happen. In case of disputes it is often possible to trade an agreed reference for agreement not to got to tribunal. Plus many prospective employers (who are employees themselves at some point) appreciate that there are two sides to any story and want to make their own minds up about it. If they are worth working for that is!
Are you disclosing Aspergers?