I want to implement a rewrite rule in my .htaccess file to show a specific url as a directory of my server. See the code below I written,

RewriteRule ^(.*)/$ ?page=$1 [NC]

This will rewrites urls like www.mysite.com/abc/ to www.mysite.com/index.php?page=abc.
But if I request www.mysite.com/abc then it is throwing an 404 error.
How can I write a rewrite rule which will match www.mysite.com/abc and www.mysite.com/abc/ both?

Edit:
My current .htaccess file (After Litso's answer's 3rd revision) is like below:

##

ErrorDocument 401 /index.php?error=401
ErrorDocument 400 /index.php?error=400
ErrorDocument 403 /index.php?error=403
ErrorDocument 500 /index.php?error=500
ErrorDocument 404 /index.php?error=404

DirectoryIndex index.htm index.html index.php

RewriteEngine on
RewriteBase /
Options +FollowSymlinks
RewriteRule ^(.+)\.html?$ $1.php
RewriteCond !-d
RewriteRule ^(.*)/$ ?page=$1 [NC,L]
RewriteCond %{REQUEST_URI} !index.php
RewriteRule ^(.*)$ ?page=$1 [NC,L]

##