You want to use hyperref only for \url and \href commands, but not all other features like PDF bookmarks, section links etc., is it? There is the url package for once. Not sure if it also provides \href. – 

Martin Scharrer

\usepackage[implicit=false]{hyperref}

Not sure exactly what you are asking, but does this help:

   \documentclass{article}

    \usepackage{hyperref} 

    \begin{document}

    Here is a url: \url{http://www.google.com}

    Here is the \href{http://www.google.com}{google site}.

    \end{document}