Search Results

Search found 7625 results on 305 pages for 'scraper sites'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 62/305 | < Previous Page | 58 59 60 61 62 63 64 65 66 67 68 69  | Next Page >

  • Servers/Websites Keep Going Down

    - by Tyler Johnson
    Okay, I'm a noobie. I know how to build and compose a website, but I have no idea what I'm doing when it comes to servers and server commands, etc. I've recently had a problem with all of my sites on our servers going down all at once and then I have to go in and reboot the server for them to come up again. At first this was annoying, but now it is becoming agonizing as it now takes 3-4 reboots for the websites to come back up. I contacted support for my hosting, but they are not being very helpful. They just keep telling me what the issue might be and basically telling me that I'm going to have to look into it and figure it out, which really isn't possible since I know nothing. Anyway, here are the things they said were possible reasons: They said I have "strange logs" in my Apache webserver log, error: sh: fetch: command not found. My php.ini memory limit is: 256M which is very high. It should be 32M or 64M. Server is reaching Max Clients, meaning we have more than 150 visitors at a time. (They supposedly "fixed" this, but the sites/server are still going down) I have some Wordpress sites with plugins getting errors like: PHP Warning: pack(): Type H: illegal hex digit G in... PHP Fatal error: Cannot use object of type stdClass as array in... PHP Fatal error: Maximum execution time of 30 seconds exceeded in... PHP Fatal error: Call to undefined function file_exists() in... PHP Parse error: syntax error, unexpected '<' I know that's a lot, but I really am at wits end and have no idea what to do now. If anyone could maybe give me some advice or point me in the right direction I would greatly appreciate it! Thanks! Oh, and here are the specs for my server: RAM: 2048MB CPU Shares: 40 Primary Disk: 50GB Data Transfer: 75GB Port Speed: 5Mbps Type: Linux

    Read the article

  • Nginx Server Block Not Working? - Already running other vhosts just this one not working

    - by daveaspinall
    Im running a Debian 6 LEMP server with multiple virtual hosts and everything has been fine for 5 or so sites. But I've just tried adding another but for some reason it's just not working. By not working I mean in Chrome I get the "Oops! Google Chrome could not connect to subdomain.domain.net" error. I've changed the domain for security to subdomain.example.com and the IP is masked. Hosts file (I have multiple sub domains): xxx.xxx.xx.xxx *.example.com *.example Server Block: server { listen 80; server_name subdomain.example.com; access_log /srv/www/subdomain.example.com/logs/access.log; error_log /srv/www/subdomain.example.com/logs/error.log; root /srv/www/subdomain.example.com/public_html; location / { index index.html index.htm index.php; } location ~ \.php$ { include fastcgi_params; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; } } I've created the system link to the file in the /etc/nginx/sites-enabled/ directory and restarted/reloaded nginx. DNS seems fine: # ping -c 2 subdomain PING subdomain.example.com (xxx.xxx.xx.xxx) 56(84) bytes of data. 64 bytes from www.example.com (xxx.xxx.xx.xxx): icmp_req=1 ttl=64 time=0.035 ms 64 bytes from www.example.com (xxx.xxx.xx.xxx): icmp_req=2 ttl=64 time=0.048 ms Checking the file with cURL works: # curl http://subdomain.example.com HTML - OK Emptied browser cache but still no dice. Anything I'm missing? Like I mentioned, I have a few sites running fine on the server currently so php-fpm etc etc are working. Any help would be much appreciated! Cheers, Dave

    Read the article

  • Script to mirror MS SQL Server databases between 2 servers

    - by David W
    Hi I have about 200 sites each of which have 2 servers running MSSQL (2k5 at some sites, 2k8 at others) One server is production and the other is primarily there as a backup. We're rebuilding all of these servers this year and as part of that we will have to set up mirroring for ... a lot ... of databases. Some of these sites have 45 databases so mirroring them manually is going to be a huge pain. I was going to write a batch script which uses SQLCMD to backup the database and log, copies to the secondary server, restores the backup and log with norecovery, creates the endpoints and sets the partner. This in itself isn't too complicated, but i'd love to see what other people have done as i'm not very confident in catching errors using the process i've outlined above. I've seen Tools to manage sql 2008 database mirroring? Which looks really good, but the formatting is jumbled and I can't get it to work. If anyone has any other scripts they've written and are willing to share I'd be eternally grateful. Ideally I'd love to be able to use a script to ensure there are matching endpoints (same ports) on both servers, backup the database, backup the log, copy the backups to second server, restore database and log with norecovery, set the partners on both servers, and somehow confirm that the databases are linked and synchronized. Well, thanks for reading :)

    Read the article

  • Seperate external and intranet portals using the same functions .htaccess

    - by jezzipin
    We are currently struggling with setting up rules for a .htaccess file for a website built upon our company product. The product is built using PLSQL and procedures can be accessed using URLs. We use this functionality to present different options to our users. These options can be injected into HTML pages using replacement tags. So, the tag [user_menu] is always replaced with: /wd_portal_cand.menu?p_web_site_id={variable1}&p_candidate_id={variable2} for external sites and /intranet/wd_portal_cand.menu?p_web_site_id={variable1}&p_candidate_id={variable2} for internal sites. The issue we are having is twofold. We need to write our .htaccess rules so that the user can access the functionality whether they are internal or external. So, the links should work as follows: http://www.example.com/wd_portal_cand.menu?p_web_site_id={variable1}&p_candidate_id={variable2} or http://www.example.com/internal/wd_portal_cand.menu?p_web_site_id={variable1}&p_candidate_id={variable2} This is the other problem. As you can see for the internal link above, the procedure needs to be prefixed with internal instead or intranet. We cannot change this in our standard tags as this will affect other sites so we need to achieve this also using htaccess. Could anyone assist with this issue? I apologise if this is brief or confusing but it's something i've never done before and have been given the task of doing. I apologise for the lack of code that will be posted above however I am a front end developer and have been left to make these changes having no prior experience of .htaccess to please bare with me.

    Read the article

  • Pulling application updates from closest server?

    - by Mike Morris
    Setup: 6 Major Sites with Server 2003/2008 DCs doing DHCP/AD Integrated DNS, each on their own subnet. All connect back to datacenter through a 3 mbps WAN ERP server running in the datacenter, accessed by clients at all sites Currently, when we update the software, I manually push a copy of the updated client/config files down to each DC. I have a script that we run on each PC to update the clients. It determines what subnet the PC is on, and pulls the software from that DC. It's messy, but it works. The client has an autoupdate feature, but it'll only pull from the application server (which is housed in the datacenter, over the 3 meg link). It takes forever, since the updates are not "patches" but a full version of the client, even for minor upgrades (bad design). After the most recent patch, you can configure the clients to pull from a different server. Unfortunately, it is the same for all clients. Is there some kind of DNS magic I can use to pull from the local server? For instance, if I tell the clients their update server is ERPUPDATE, can I have their local DNS server return a different IP for ERPUPDATE than the other sites? Example: Client 1 is at site A, client 2 is at site b. They each run the software and a version change is detected. As per the config files, the clients look to ERPUPDATE for their updated client. Client 1 queries DNS for the IP of ERPUPDATE at its current location (site A) DNS at site A returns 192.1.1.5 Client 1 pulls update from 192.1.1.5 Client 2 queries DNS for the IP of ERPUPDATE at its current location (site B) DNS at site B returns 192.1.2.5 Client 2 pulls update from 192.1.2.5 Excuse the poor explanation, I worked 61 hours over the weekend and haven't completely rebounded. I'll be happy to clarify if needed!

    Read the article

  • How to Access User Directory shared by Apache on OS X Mountain Lion?

    - by schluchc
    When trying to access the local user web page on localhost/~username, I get a "403 Forbidden". The system web page in /Library/WebServer/Documents is accessible on localhost/ though, so I assume Apache is working fine. I know that this problem has been discussed several times, also on superuser. I implemented and checked all I could find, but I still couldn't solve the problem and would be glad if someone had a suggestion for this particular case: sudo apachectl -t returns Syntax OK. I have a username.conf file in /etc/apache2/users/: <Directory "/Users/username/Sites/"> Options Indexes MultiViews FollowSymLinks AllowOverride AuthConfig Limit Order allow,deny Allow from all </Directory> as proposed here [SuperUser] and in several other tutorials. The permissions of the username.conf file are -rw-r--r-- root wheel, as they should be. The httpd.conf is unchanged and therefore contains the line Include /private/etc/apache2/extra/httpd-userdir.conf. That file in turn contains UserDir Sites Include /private/etc/apache2/users/*.conf <IfModule bonjour_module> RegisterUserSite customized-users </IfModule> So the httpd*.conf files should be ok. The permissions of /Users/username/Sites is drwxr-xr-x 10 username staff and -rw-r--r--@ 1 username staff for the index.html. In the error log I simply get a [Sun Nov 25 22:14:32 2012] [error] [client 127.0.0.1] (13)Permission denied: access to /~username/ denied. And yes, after each change I did the sudo apachectl restart. Any help no how to solve the problem or how to further analyze it would be highly appreciated!

    Read the article

  • Adventures in Drupal multisite config with mod_rewrite and clean urls

    - by moexu
    The university where I work is planning to offer Drupal hosting to staff/faculty who want a Drupal site. We've set up Drupal multisite with clean urls and it's mostly working except for some weird redirects. If you have two sites where one is a substring of the other then you'll randomly be redirected to the other site. I tracked the problem to how mod_rewrite does path matching, so with a config file like this: RewriteCond %{REQUEST_URI} ^/drupal RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^(.*)$ /drupal/index.php?q=$1 [last,qsappend] RewriteCond %{REQUEST_URI} ^/drupaltest RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^(.*)$ /drupaltest/index.php?q=$1 [last,qsappend] /drupaltest will match the /drupal line and all of the links on the /drupaltest page will be rewritten to point to /drupal. If you put the end of string character ($) at the end of each rewrite condition then it will always match on the correct site and the links will always be rewritten correctly. That breaks down as soon as a user logs in though because the query string is appended to the url so just the base url will no longer match. You can also fix the problem by ordering the sites in the config file so that the smallest substring will always be last. I suggested storing all of the sites in a table and then querying, sorting, and rewriting the config file every time a Drupal site is requested so that we could guarantee the order. The system administrator thought that was kludgy and didn't address the root problem. Disabling clean urls should also fix the problem but the users really want them so I'd prefer to keep them if possible. I think we could also fix it by using an .htaccess file in each site to handle the clean url rewriting but that also seems suboptimal since it will generate a higher load on the server and the server is intended to host the majority of the university's external facing web content. Is there some magic I can do with mod_rewrite to get it to work? Would another solution be better? Am I doing something the wrong way to begin with?

    Read the article

  • Multiple SSL certificates on Apache using multiple public IPs - not working

    - by St. Even
    I need configure multiple SSL certificates on a single Apache server. I already know that I need multiple external IP addresses as I cannot use SNI (only running Apache 2.2.3 on this server). I assumed that I had everything configured correctly, unfortunately things are not working as they should (or maybe I should say, as I expected them to work)... In my httpd.conf I have: NameVirtualHost *:80 NameVirtualHost *:443 Lets say my public IP is 12.0.0.1 and my private IP is 192.168.0.1. When I use the public IP in my vhost my default website is being shown instead the one defined in my vhost, e.g.: <VirtualHost 12.0.0.1:443> ServerAdmin [email protected] ServerName blablabla.site.com DocumentRoot /data/sites/blablabla.site.com ErrorLog /data/sites/blablabla.site.com-error.log #CustomLog /data/sites/blablabla.site.com-access.log common SSLEngine On SSLCertificateFile /etc/httpd/conf/ssl/blablabla.site.com.crt SSLCertificateKeyFile /etc/httpd/conf/ssl/blablabla.site.com.key SSLCertificateChainFile /etc/httpd/conf/ssl/blablabla.site.com.ca-bundle <Location /> SSLRequireSSL On SSLVerifyDepth 1 SSLOptions +StdEnvVars +StrictRequire </Location> </VirtualHost> When I use the private IP in my vhost everything works as it should (the website defined in my vhost is being shown), e.g.: <VirtualHost 192.168.0.1:443> ...same as above... </VirtualHost> My server is listening on all interfaces: [root@grbictwebp02 httpd]# netstat -tulpn | grep :443 tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 5585/httpd What am I doing wrong? If I cannot get this to work I cannot continue to add the second SSL certificate on the other public IP... If more information is required just let me know!

    Read the article

  • Viewing local websites on my iOS device over Wi-fi

    - by John
    Trying to view some local html/css/js files in a mobile browser on my iOS device. Thought maybe file-sharing would be an option, and is, but I'm not completely satisfied with it. Any time I try to do the following an error occurs. Web sharing is on and available at http://192.168.1.101/~user but I have to manually copy the files in. If I try to symlink a folder in so that the address could be viewed at ''~user/some_dir by issuing $ ln -s /Users/user/dev/some_dir ~/Sites/ then I get a 403 forbidden error. I've tried to remedy this by modifying a user.conf file in /private/etc/apache2/ and using the following syntax: <Directory "/Users/user/Sites/"> Options Indexes MultiViews SymLinks AllowOverride None Order allow,deny Allow from all </Directory> but nope, still doesn't work. I get a 403 error. If I try to symlink each individual file in instead of using a directory as a sub-directory, same error. Any help would be greatly appreciated! I'd just like to symlink directories into the ~/Sites one and browse them on my iOS device over wifi. I'm on OS X 10.7 Lion trying to connect with iOS 5.

    Read the article

  • Google Chrome 'ruined' after someone else logged in

    - by MHJ96
    Google Chrome was the default browser that came with my laptop. I have a Google account which I was logged into Chrome with. Someone else logged into my chrome using their account which has resulted in everything being lost. I logged in again to find my theme, bookmarks, history, most visited sites were gone and now instead of 'piling up' under the symbol already pinned to the Windows 7 task bar it opens a second symbol on the taskbar and 'piles up' under that instead which has never happened before. I have tried unpinning and repinning which didn't work. I have tried syncing my account numerous times to no avail, I have searched high and low on the Chrome forums for any kind of answer. I have tried accessing that thing in documents to try and recover my bookmarks but I can't find them (I had hidden files enabled etc etc). I really really want it back how it was as I had a lot of bookmarked sites and quick access to sites and everything was how i used it, and I hate that it now opens a new icon on the taskbar.

    Read the article

  • Referer is passed from HTTPS to HTTP in some cases... How?

    - by ravisorg
    In theory browsers do not pass on referer information from HTTPS to HTTP sites. And in my experience this has always been true. But I just found an exception, and I want to understand why it works so I can use it as well. Search for "what is my referer" on https://www.google.ca/ eg: https://www.google.ca/search?q=what+is+my+referer There are a few sites that will show referer. They all seem to "work" when they shouldn't. For example, click the www.whatismyreferer.com one. I get: Your referer: https://www.google.ca/ Note that sometimes, rarely, I get "no referer" as the result. Go back and click the link again and it'll "work" the next time. This should not happen. www.whatismyreferer.com is a non-HTTPS site. The referer header should not be being passed, but it is. What's going on here, and how can I do the same from my HTTPS site to the HTTP sites I'm linking to?

    Read the article

  • News feed APIs for general news

    - by dassouki
    I'm building a database + tool that scours news feeds for a certain term. For example "food poisoning from nuts". I want to scour social media sites, news sites, major news aggregators, etc... for that term. Question 1: What are some of the news aggregator APIs out there? Question 2: How Would you go about coding and receiving only the latest news from the API?

    Read the article

  • Intermittent PolicyException: Execution permission cannot be acquired.

    - by Aaron Maenpaa
    We are intermittently seeing the following exception shortly after an App Pool recycle in an ASP.NET application: System.Configuration.ConfigurationErrorsException: Could not load file or assembly 'Microsoft.Web.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. Failed to grant permission to execute. (Exception from HRESULT: 0x80131418) ---> System.IO.FileLoadException: Could not load file or assembly 'Microsoft.Web.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. Failed to grant permission to execute. (Exception from HRESULT: 0x80131418) File name: 'Microsoft.Web.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' ---> System.Security.Policy.PolicyException: Execution permission cannot be acquired. at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Boolean checkExecutionPermission) at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Int32& securitySpecialFlags, Boolean checkExecutionPermission) at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) at System.Reflection.Assembly.Load(String assemblyString) at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective) The specific DLL that fails to load varies from incident to incident, but is always one referenced by the main assembly. We're running on ASP.NET 3.5 on Windows Server 2008. This seems to happen in batches affecting some but not all of sites on the same App Pool. We have a large number of sites all running the same code. Once a site has failed to load a DLL it throws up a Yellow Screen of Death until the next App Pool recycle. We haven't been able to reproduce this behavior and the sites seem to work fine for days or weeks at a time (and many App Pool recycles) before failing. Has anybody else seen similar behavior? Update: We've tried reproducing the failure by setting up a few hundred sites and writing a script to hit them repeatedly while recycling the App Pool once every couple of minutes and were unable to accomplish much other than loading down the server's CPU for a few days straight. We then tried messing (locking one of the DLLs, changing the file permissions) with the copies of the DLLs that ASP.NET makes and managed to reproduce similar behavior but not the same exception. Does anybody have any ideas on how to adjust the security policy to get it to throw a System.Security.Policy.PolicyException: Execution permission cannot be acquired. when loading a specific DLL?

    Read the article

  • Sharepoint and its template/master pages

    - by jhuang
    I'm trying to learn how to customize a sharepoint site. I've seen a lot of examples and they all seem to have one thing in common, the navigation are all the same. Ie the left nav with the different pages and docs. top nav with mostly links to sub-sites or other sites. Does anyone know of a good tutorial or example i can grab that has a horizontal layout of the left nav? i haven't found any.

    Read the article

  • php framework with sqlite, orm, i18n, l10n search addon

    - by ddorian
    im trying to find a php framework to build small,multilingual sites. do you know a php framework with support for: 1.sqlite (it will be little sites so no performance problem and good for copy-paste from development to production) 2.orm 3.i18n & l10n 4.easy search addon 5.ability to just copy-paste no need to change config for going from devel machine to production and if you know cms with those features put it too thank you

    Read the article

  • Where can you find fun/educational programming challenges?

    - by tj9991
    I've searched around for different challenge sites, and most of them seem to be geared towards difficulty in problem solving logically, rather than trying to use your language of choice to do something you haven't used it for. Their center is around mathematics rather than function design. Some kind of point system for correctly solving challenges, or solving them the most efficient/smallest would be neat as well. Listed sites Project Euler TopCoder UVa Online Judge Challenges with Python Google Code Jam Programming Challenges Less Than Dot ACM's Programing Contest archive USACO problems ITA Software's puzzle page Refactor My Code Ruby Quiz

    Read the article

  • Minimum permission required to access Site Columns page and edit site columns

    - by ParanoidMike
    We've started to adopt SharePoint 2010, and are starting to manually migrate content from SharePoint 2007 sites to new sites we're rebuilding from scratch in SP2010. One of the things we previously had supported was to delegate responsibility for managing some of our site columns to a member of the team. The team member is not familiar with SharePoint internals, and doesn't want the responsibility of full permissions to the site and all its objects. We're now trying to figure out what the minimum permission is that we need to grant our team member, so they can continue to edit (& propagate) the content of the site columns we've defined. Permissions he currently has (which are obviously insufficient): Site permissions (according to _layouts/user.aspx): Read, Contribute, Manage Lists Permissions for specific objects in the site (according to _layouts/people.aspxMembershipGroupId=xxx, then choosing Settings, View Group Permissions): server/sites/[sitename]: Contribute server/sites/[sitename]/Lists/[a list with columns that inherit from site columns]: "Contribute No Delete" Note: the "Contribute No Delete" permission is a custom permission I designed by copying the SharePoint-native "Contribute" permission set and deselecting the Delete permission. The "Manage Lists" permission is a custom permission I designed that includes the following specific permissions: (List Permissions) Manage Lists, View Items; (Site Permissions) View Pages, Open. Operations that are throwing access denied errors: _layouts/mngfield.aspx: SharePoint returns the "Error: Access Denied" dialog, and provides three clickable options: "Sign in as a different user", "Request access", and "Go back to site" _layouts/fldedit.aspx?field=Level%5Fx0020%5F3 [one of the site columns we've defined]: can load the page and type in changes to the textboxes "...but when I press OK (save changes) I get the same message above." When our team member clicks the "Request access" link, the email I receive sends me to a page that recommends that I grant the user membership in the "[sitename] Users" group - of which he's already a member. So while SP2010 tries to request access, it doesn't actually direct me to either (a) a valid group that has the correct permissions or (b) the specific object to which I need to grant our team member access. Also note: on the SP2007 (MOSS) site (where our team member was successful in managing Site Column edits), they had dozens of additional permissions throughout the site that we do not wish to blindly re-allocate in SP2010 until we know they're necessary. Any help anyone can provide would be greatly appreciated.

    Read the article

  • Django Admin Actions missing

    - by Andrew C
    One of my Django sites is missing the Django Admin Action bar shown here: http://docs.djangoproject.com/en/dev/ref/contrib/admin/actions/#ref-contrib-admin-actions There is no checkbox next to each row and no Action select box near the top of the page. This is happening on every model. I have several sites running Django 1.1, and they all show the Admin Actions, so it feels like a local configuration issue. Anyone seen this before?

    Read the article

  • Palm OS 5 development tools

    - by Jean Paul
    Hello. A few time ago I make a question about the Palm OS 5 development tools. Here I am again. I have seached a lot in Google and in many developer sites but all the links are broken and the sites are too old. Does anyone know a real tool in any OS (The best wold be for Windows or Linux) so I can develop, test and deploy software for Palm OS 5???? Thanks!!!!

    Read the article

  • MSDN Subscription Question for the lone developer

    - by BrianLy
    I'm looking to get an MSDN Subscription and I see a number of sites offering 2 year subscriptions versions. Are these sites offering a regular version that I can buy or are they for Software Assurance customers only? I don't want to buy one and find out I cannot activate it because I'm not associated with a company that has SA.

    Read the article

  • So now that Django 1.2 is officially out...

    - by Fedor
    Since I have Django 1.1x on my Debian setup - how can I use virtualenv or similar and not have it mess up my system's default django version which in turn would break all my sites? Detailed instructions or a great tutorial link would very much be appreciated - please don't offer vague advice since I'm still a noob. Currently I store all my django projects in ~/django-sites and I am using Apache2 + mod_wsgi to deploy.

    Read the article

  • IIS Manager IIS6 Rights Management on Windows Server 2003

    - by Chris
    We are using a dedicated hosting provider to host several of our public websites. There are two development teams/divisions within our company who each maintain public websites on these same servers. Is there a way through ACLs or some sort of rights management (e.g. individual logins) that I would only be able to see and maintain only my public sites and the other development team could only see and maintain their sites? Thanks! Chris

    Read the article

  • mysql index optimization for a table with multiple indexes that index some of the same columns

    - by Sean
    I have a table that stores some basic data about visitor sessions on third party web sites. This is its structure: id, site_id, unixtime, unixtime_last, ip_address, uid There are four indexes: id, site_id/unixtime, site_id/ip_address, and site_id/uid There are many different types of ways that we query this table, and all of them are specific to the site_id. The index with unixtime is used to display the list of visitors for a given date or time range. The other two are used to find all visits from an IP address or a "uid" (a unique cookie value created for each visitor), as well as determining if this is a new visitor or a returning visitor. Obviously storing site_id inside 3 indexes is inefficient for both write speed and storage, but I see no way around it, since I need to be able to quickly query this data for a given specific site_id. Any ideas on making this more efficient? I don't really understand B-trees besides some very basic stuff, but it's more efficient to have the left-most column of an index be the one with the least variance - correct? Because I considered having the site_id being the second column of the index for both ip_address and uid but I think that would make the index less efficient since the IP and UID are going to vary more than the site ID will, because we only have about 8000 unique sites per database server, but millions of unique visitors across all ~8000 sites on a daily basis. I've also considered removing site_id from the IP and UID indexes completely, since the chances of the same visitor going to multiple sites that share the same database server are quite small, but in cases where this does happen, I fear it could be quite slow to determine if this is a new visitor to this site_id or not. The query would be something like: select id from sessions where uid = 'value' and site_id = 123 limit 1 ... so if this visitor had visited this site before, it would only need to find one row with this site_id before it stopped. This wouldn't be super fast necessarily, but acceptably fast. But say we have a site that gets 500,000 visitors a day, and a particular visitor loves this site and goes there 10 times a day. Now they happen to hit another site on the same database server for the first time. The above query could take quite a long time to search through all of the potentially thousands of rows for this UID, scattered all over the disk, since it wouldn't be finding one for this site ID. Any insight on making this as efficient as possible would be appreciated :) Update - this is a MyISAM table with MySQL 5.0. My concerns are both with performance as well as storage space. This table is both read and write heavy. If I had to choose between performance and storage, my biggest concern is performance - but both are important. We use memcached heavily in all areas of our service, but that's not an excuse to not care about the database design. I want the database to be as efficient as possible.

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 58 59 60 61 62 63 64 65 66 67 68 69  | Next Page >