Swansea head for Hull this weekend in upbeat mood after backing up their morale-boosting draw at Arsenal with a thumping win over Norwich last Saturday.
After the Tigers clash, Swansea face home games with Chelsea, Aston Villa and Southampton plus away trips to Newcastle and Sunderland.
And Williams believes that Swansea should have bolder ambitions than mere survival.
"What we don't want to focus on is getting to 37 or 38 points and then stopping," the Swansea captain said.
"We're trying not to lose again this season, to win as many as we can and to just keep going. It's in our hands.
"We're not saying: 'Let's go for two more wins'.
"Forget that. We've got six games left so why can't we try to win them all? If we can't win them, let's not lose.
"That's the way we're looking at it.
"It's a bit bullish to say that after we've just drawn one game and won one game, but that's the way we feel.
"We've got to set our standards high and get back to where we have been in the last two years.
"We want to finish the season strong and take the momentum into next season."
Swansea boss Garry Monk has admitted the club's general form this term has not been good enough.
It is been a difficult year, with Europa League involvement perhaps taking its toll on Swansea's domestic performances.
The Liberty club began the season feeling they had assembled their strongest ever squad, yet the decline in form compared to last season saw Michael Laudrup jettisoned in February.
Monk's impact has been positive — yet Swansea have continued to find it hard work to get results.
But after a bright week, Swansea will be full of confidence when they run out at the KC Stadium on Saturday.
"The feelgood factor is back," Williams added.
"Look at Wayne Routledge — there can't be many that work harder than him.
"The application he's shown is unbelievable.
"Wilfried Bony is unplayable at times, Jonathan de Guzman is going out and doing a shift in a position that isn't his normal position — he's doing it for the team.
"You look all across the team. Substitutes are coming on and putting their bodies on the line.
"We know that's what we need. We've said it for weeks, but now we're actually showing it, which is great."