Search Results

Search found 55 results on 3 pages for 'datagram'.

Page 3/3 | < Previous Page | 1 2 3 

• Can I avoid a threaded UDP socket in Pyton dropping data?

  - by 666craig

  First off, I'm new to Python and learning on the job, so be gentle! I'm trying to write a threaded Python app for Windows that reads data from a UDP socket (thread-1), writes it to file (thread-2), and displays the live data (thread-3) to a widget (gtk.Image using a gtk.gdk.pixbuf). I'm using queues for communicating data between threads. My problem is that if I start only threads 1 and 3 (so skip the file writing for now), it seems that I lose some data after the first few samples. After this drop it looks fine. Even by letting thread 1 complete before running thread 3, this apparent drop is still there. Apologies for the length of code snippet (I've removed the thread that writes to file), but I felt removing code would just prompt questions. Hope someone can shed some light :-) import socket import threading import Queue import numpy import gtk gtk.gdk.threads_init() import gtk.glade import pygtk class readFromUDPSocket(threading.Thread): def __init__(self, socketUDP, readDataQueue, packetSize, numScans): threading.Thread.__init__(self) self.socketUDP = socketUDP self.readDataQueue = readDataQueue self.packetSize = packetSize self.numScans = numScans def run(self): for scan in range(1, self.numScans + 1): buffer = self.socketUDP.recv(self.packetSize) self.readDataQueue.put(buffer) self.socketUDP.close() print 'myServer finished!' class displayWithGTK(threading.Thread): def __init__(self, displayDataQueue, image, viewArea): threading.Thread.__init__(self) self.displayDataQueue = displayDataQueue self.image = image self.viewWidth = viewArea[0] self.viewHeight = viewArea[1] self.displayData = numpy.zeros((self.viewHeight, self.viewWidth, 3), dtype=numpy.uint16) def run(self): scan = 0 try: while True: if not scan % self.viewWidth: scan = 0 buffer = self.displayDataQueue.get(timeout=0.1) self.displayData[:, scan, 0] = numpy.fromstring(buffer, dtype=numpy.uint16) self.displayData[:, scan, 1] = numpy.fromstring(buffer, dtype=numpy.uint16) self.displayData[:, scan, 2] = numpy.fromstring(buffer, dtype=numpy.uint16) gtk.gdk.threads_enter() self.myPixbuf = gtk.gdk.pixbuf_new_from_data(self.displayData.tostring(), gtk.gdk.COLORSPACE_RGB, False, 8, self.viewWidth, self.viewHeight, self.viewWidth * 3) self.image.set_from_pixbuf(self.myPixbuf) self.image.show() gtk.gdk.threads_leave() scan += 1 except Queue.Empty: print 'myDisplay finished!' pass def quitGUI(obj): print 'Currently active threads: %s' % threading.enumerate() gtk.main_quit() if __name__ == '__main__': # Create socket (IPv4 protocol, datagram (UDP)) and bind to address socketUDP = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) host = '192.168.1.5' port = 1024 socketUDP.bind((host, port)) # Data parameters samplesPerScan = 256 packetsPerSecond = 1200 packetSize = 512 duration = 1 # For now, set a fixed duration to log data numScans = int(packetsPerSecond * duration) # Create array to store data data = numpy.zeros((samplesPerScan, numScans), dtype=numpy.uint16) # Create queue for displaying from readDataQueue = Queue.Queue(numScans) # Build GUI from Glade XML file builder = gtk.Builder() builder.add_from_file('GroundVue.glade') window = builder.get_object('mainwindow') window.connect('destroy', quitGUI) view = builder.get_object('viewport') image = gtk.Image() view.add(image) viewArea = (1200, samplesPerScan) # Instantiate & start threads myServer = readFromUDPSocket(socketUDP, readDataQueue, packetSize, numScans) myDisplay = displayWithGTK(readDataQueue, image, viewArea) myServer.start() myDisplay.start() gtk.gdk.threads_enter() gtk.main() gtk.gdk.threads_leave() print 'gtk.main finished!'

  Read the article

• Network communications mechanisms for SQL Server

  - by Akshay Deep Lamba

  Problem I am trying to understand how SQL Server communicates on the network, because I'm having to tell my networking team what ports to open up on the firewall for an edge web server to communicate back to the SQL Server on the inside. What do I need to know? Solution In order to understand what needs to be opened where, let's first talk briefly about the two main protocols that are in common use today: TCP - Transmission Control Protocol UDP - User Datagram Protocol Both are part of the TCP/IP suite of protocols. We'll start with TCP. TCP TCP is the main protocol by which clients communicate with SQL Server. Actually, it is more correct to say that clients and SQL Server use Tabular Data Stream (TDS), but TDS actually sits on top of TCP and when we're talking about Windows and firewalls and other networking devices, that's the protocol that rules and controls are built around. So we'll just speak in terms of TCP. TCP is a connection-oriented protocol. What that means is that the two systems negotiate the connection and both agree to it. Think of it like a phone call. While one person initiates the phone call, the other person has to agree to take it and both people can end the phone call at any time. TCP is the same way. Both systems have to agree to the communications, but either side can end it at any time. In addition, there is functionality built into TCP to ensure that all communications can be disassembled and reassembled as necessary so it can pass over various network devices and be put together again properly in the right order. It also has mechanisms to handle and retransmit lost communications. Because of this functionality, TCP is the protocol used by many different network applications. The way the applications all can share is through the use of ports. When a service, like SQL Server, comes up on a system, it must listen on a port. For a default SQL Server instance, the default port is 1433. Clients connect to the port via the TCP protocol, the connection is negotiated and agreed to, and then the two sides can transfer information as needed until either side decides to end the communication. In actuality, both sides will have a port to use for the communications, but since the client's port is typically determined semi-randomly, when we're talking about firewalls and the like, typically we're interested in the port the server or service is using. UDP UDP, unlike TCP, is not connection oriented. A "client" can send a UDP communications to anyone it wants. There's nothing in place to negotiate a communications connection, there's nothing in the protocol itself to coordinate order of communications or anything like that. If that's needed, it's got to be handled by the application or by a protocol built on top of UDP being used by the application. If you think of TCP as a phone call, think of UDP as a postcard. I can put a postcard in the mail to anyone I want, and so long as it is addressed properly and has a stamp on it, the postal service will pick it up. Now, what happens it afterwards is not guaranteed. There's no mechanism for retransmission of lost communications. It's great for short communications that doesn't necessarily need an acknowledgement. Because multiple network applications could be communicating via UDP, it uses ports, just like TCP. The SQL Browser or the SQL Server Listener Service uses UDP. Network Communications - Talking to SQL Server When an instance of SQL Server is set up, what TCP port it listens on depends. A default instance will be set up to listen on port 1433. A named instance will be set to a random port chosen during installation. In addition, a named instance will be configured to allow it to change that port dynamically. What this means is that when a named instance starts up, if it finds something already using the port it normally uses, it'll pick a new port. If you have a named instance, and you have connections coming across a firewall, you're going to want to use SQL Server Configuration Manager to set a static port. This will allow the networking and security folks to configure their devices for maximum protection. While you can change the network port for a default instance of SQL Server, most people don't. Network Communications - Finding a SQL Server When just the name is specified for a client to connect to SQL Server, for instance, MySQLServer, this is an attempt to connect to the default instance. In this case the client will automatically attempt to communicate to port 1433 on MySQLServer. If you've switched the port for the default instance, you'll need to tell the client the proper port, usually by specifying the following syntax in the connection string: <server>,<port>. For instance, if you moved SQL Server to listen on 14330, you'd use MySQLServer,14330 instead of just MySQLServer. However, because a named instance sets up its port dynamically by default, the client never knows at the outset what the port is it should talk to. That's what the SQL Browser or the SQL Server Listener Service (SQL Server 2000) is for. In this case, the client sends a communication via the UDP protocol to port 1434. It asks, "Where is the named instance?" So if I was running a named instance called SQL2008R2, it would be asking the SQL Browser, "Hey, how do I talk to MySQLServer\SQL2008R2?" The SQL Browser would then send back a communications from UDP port 1434 back to the client telling the client how to talk to the named instance. Of course, you can skip all of this of you set that named instance's port statically. Then you can use the <server>,<port> mechanism to connect and the client won't try to talk to the SQL Browser service. It'll simply try to make the connection. So, for instance, is the SQL2008R2 instance was listening on port 20080, specifying MySQLServer,20080 would attempt a connection to the named instance. Network Communications - Named Pipes Named pipes is an older network library communications mechanism and it's generally not used any longer. It shouldn't be used across a firewall. However, if for some reason you need to connect to SQL Server with it, this protocol also sits on top of TCP. Named Pipes is actually used by the operating system and it has its own mechanism within the protocol to determine where to route communications. As far as network communications is concerned, it listens on TCP port 445. This is true whether we're talking about a default or named instance of SQL Server. The Summary Table To put all this together, here is what you need to know: Type of Communication Protocol Used Default Port Finding a SQL Server or SQL Server Named Instance UDP 1434 Communicating with a default instance of SQL Server TCP 1433 Communicating with a named instance of SQL Server TCP * Determined dynamically at start up Communicating with SQL Server via Named Pipes TCP 445

  Read the article

• Computer Networks UNISA - Chap 10 &ndash; In Depth TCP/IP Networking

  - by MarkPearl

  After reading this section you should be able to Understand methods of network design unique to TCP/IP networks, including subnetting, CIDR, and address translation Explain the differences between public and private TCP/IP networks Describe protocols used between mail clients and mail servers, including SMTP, POP3, and IMAP4 Employ multiple TCP/IP utilities for network discovery and troubleshooting Designing TCP/IP-Based Networks The following sections explain how network and host information in an IPv4 address can be manipulated to subdivide networks into smaller segments. Subnetting Subnetting separates a network into multiple logically defined segments, or subnets. Networks are commonly subnetted according to geographic locations, departmental boundaries, or technology types. A network administrator might separate traffic to accomplish the following… Enhance security Improve performance Simplify troubleshooting The challenges of Classful Addressing in IPv4 (No subnetting) The simplest type of IPv4 is known as classful addressing (which was the Class A, Class B & Class C network addresses). Classful addressing has the following limitations. Restriction in the number of usable IPv4 addresses (class C would be limited to 254 addresses) Difficult to separate traffic from various parts of a network Because of the above reasons, subnetting was introduced. IPv4 Subnet Masks Subnetting depends on the use of subnet masks to identify how a network is subdivided. A subnet mask indicates where network information is located in an IPv4 address. The 1 in a subnet mask indicates that corresponding bits in the IPv4 address contain network information (likewise 0 indicates the opposite) Each network class is associated with a default subnet mask… Class A = 255.0.0.0 Class B = 255.255.0.0 Class C = 255.255.255.0 An example of calculating  the network ID for a particular device with a subnet mask is shown below.. IP Address = 199.34.89.127 Subnet Mask = 255.255.255.0 Resultant Network ID = 199.34.89.0 IPv4 Subnetting Techniques Subnetting breaks the rules of classful IPv4 addressing. Read page 490 for a detailed explanation Calculating IPv4 Subnets Read page 491 – 494 for an explanation Important… Subnetting only applies to the devices internal to your network. Everything external looks at the class of the IP address instead of the subnet network ID. This way, traffic directed to your network externally still knows where to go, and once it has entered your internal network it can then be prioritized and segmented. CIDR (classless Interdomain Routing) CIDR is also known as classless routing or supernetting. In CIDR conventional network class distinctions do not exist, a subnet boundary can move to the left, therefore generating more usable IP addresses on your network. A subnet created by moving the subnet boundary to the left is known as a supernet. With CIDR also came new shorthand for denoting the position of subnet boundaries known as CIDR notation or slash notation. CIDR notation takes the form of the network ID followed by a forward slash (/) followed by the number of bits that are used for the extended network prefix. To take advantage of classless routing, your networks routers must be able to interpret IP addresses that don;t adhere to conventional network class parameters. Routers that rely on older routing protocols (i.e. RIP) are not capable of interpreting classless IP addresses. Internet Gateways Gateways are a combination of software and hardware that enable two different network segments to exchange data. A gateway facilitates communication between different networks or subnets. Because on device cannot send data directly to a device on another subnet, a gateway must intercede and hand off the information. Every device on a TCP/IP based network has a default gateway (a gateway that first interprets its outbound requests to other subnets, and then interprets its inbound requests from other subnets). The internet contains a vast number of routers and gateways. If each gateway had to track addressing information for every other gateway on the Internet, it would be overtaxed. Instead, each handles only a relatively small amount of addressing information, which it uses to forward data to another gateway that knows more about the data’s destination. The gateways that make up the internet backbone are called core gateways. Address Translation An organizations default gateway can also be used to “hide” the organizations internal IP addresses and keep them from being recognized on a public network. A public network is one that any user may access with little or no restrictions. On private networks, hiding IP addresses allows network managers more flexibility in assigning addresses. Clients behind a gateway may use any IP addressing scheme, regardless of whether it is recognized as legitimate by the Internet authorities but as soon as those devices need to go on the internet, they must have legitimate IP addresses to exchange data. When a clients transmission reaches the default gateway, the gateway opens the IP datagram and replaces the client’s private IP address with an Internet recognized IP address. This process is known as NAT (Network Address Translation). TCP/IP Mail Services All Internet mail services rely on the same principles of mail delivery, storage, and pickup, though they may use different types of software to accomplish these functions. Email servers and clients communicate through special TCP/IP application layer protocols. These protocols, all of which operate on a variety of operating systems are discussed below… SMTP (Simple Mail transfer Protocol) The protocol responsible for moving messages from one mail server to another over TCP/IP based networks. SMTP belongs to the application layer of the ODI model and relies on TCP as its transport protocol. Operates from port 25 on the SMTP server Simple sub-protocol, incapable of doing anything more than transporting mail or holding it in a queue MIME (Multipurpose Internet Mail Extensions) The standard message format specified by SMTP allows for lines that contain no more than 1000 ascii characters meaning if you relied solely on SMTP you would have very short messages and nothing like pictures included in an email. MIME us a standard for encoding and interpreting binary files, images, video, and non-ascii character sets within an email message. MIME identifies each element of a mail message according to content type. MIME does not replace SMTP but works in conjunction with it. Most modern email clients and servers support MIME POP (Post Office Protocol) POP is an application layer protocol used to retrieve messages from a mail server POP3 relies on TCP and operates over port 110 With POP3 mail is delivered and stored on a mail server until it is downloaded by a user Disadvantage of POP3 is that it typically does not allow users to save their messages on the server because of this IMAP is sometimes used IMAP (Internet Message Access Protocol) IMAP is a retrieval protocol that was developed as a more sophisticated alternative to POP3 The single biggest advantage IMAP4 has over POP3 is that users can store messages on the mail server, rather than having to continually download them Users can retrieve all or only a portion of any mail message Users can review their messages and delete them while the messages remain on the server Users can create sophisticated methods of organizing messages on the server Users can share a mailbox in a central location Disadvantages of IMAP are typically related to the fact that it requires more storage space on the server. Additional TCP/IP Utilities Nearly all TCP/IP utilities can be accessed from the command prompt on any type of server or client running TCP/IP. The syntaxt may differ depending on the OS of the client. Below is a list of additional TCP/IP utilities – research their use on your own! Ipconfig (Windows) & Ifconfig (Linux) Netstat Nbtstat Hostname, Host & Nslookup Dig (Linux) Whois (Linux) Traceroute (Tracert) Mtr (my traceroute) Route

  Read the article

• How to Avoid Your Next 12-Month Science Project

  - by constant

  While most customers immediately understand how the magic of Oracle's Hybrid Columnar Compression, intelligent storage servers and flash memory make Exadata uniquely powerful against home-grown database systems, some people think that Exalogic is nothing more than a bunch of x86 servers, a storage appliance and an InfiniBand (IB) network, built into a single rack. After all, isn't this exactly what the High Performance Computing (HPC) world has been doing for decades? On the surface, this may be true. And some people tried exactly that: They tried to put together their own version of Exalogic, but then they discover there's a lot more to building a system than buying hardware and assembling it together. IT is not Ikea. Why is that so? Could it be there's more going on behind the scenes than merely putting together a bunch of servers, a storage array and an InfiniBand network into a rack? Let's explore some of the special sauce that makes Exalogic unique and un-copyable, so you can save yourself from your next 6- to 12-month science project that distracts you from doing real work that adds value to your company. Engineering Systems is Hard Work! The backbone of Exalogic is its InfiniBand network: 4 times better bandwidth than even 10 Gigabit Ethernet, and only about a tenth of its latency. What a potential for increased scalability and throughput across the middleware and database layers! But InfiniBand is a beast that needs to be tamed: It is true that Exalogic uses a standard, open-source Open Fabrics Enterprise Distribution (OFED) InfiniBand driver stack. Unfortunately, this software has been developed by the HPC community with fastest speed in mind (which is good) but, despite the name, not many other enterprise-class requirements are included (which is less good). Here are some of the improvements that Oracle's InfiniBand development team had to add to the OFED stack to make it enterprise-ready, simply because typical HPC users didn't have the need to implement them: More than 100 bug fixes in the pieces that were not related to the Message Passing Interface Protocol (MPI), which is the protocol that HPC users use most of the time, but which is less useful in the enterprise. Performance optimizations and tuning across the whole IB stack: From Switches, Host Channel Adapters (HCAs) and drivers to low-level protocols, middleware and applications. Yes, even the standard HPC IB stack could be improved in terms of performance. Ethernet over IB (EoIB): Exalogic uses InfiniBand internally to reach high performance, but it needs to play nicely with datacenters around it. That's why Oracle added Ethernet over InfiniBand technology to it that allows for creating many virtual 10GBE adapters inside Exalogic's nodes that are aggregated and connected to Exalogic's IB gateway switches. While this is an open standard, it's up to the vendor to implement it. In this case, Oracle integrated the EoIB stack with Oracle's own IB to 10GBE gateway switches, and made it fully virtualized from the beginning. This means that Exalogic customers can completely rewire their server infrastructure inside the rack without having to physically pull or plug a single cable - a must-have for every cloud deployment. Anybody who wants to match this level of integration would need to add an InfiniBand switch development team to their project. Or just buy Oracle's gateway switches, which are conveniently shipped with a whole server infrastructure attached! IPv6 support for InfiniBand's Sockets Direct Protocol (SDP), Reliable Datagram Sockets (RDS), TCP/IP over IB (IPoIB) and EoIB protocols. Because no IPv6 = not very enterprise-class. HA capability for SDP. High Availability is not a big requirement for HPC, but for enterprise-class application servers it is. Every node in Exalogic's InfiniBand network is connected twice for redundancy. If any cable or port or HCA fails, there's always a replacement link ready to take over. This requires extra magic at the protocol level to work. So in addition to Weblogic's failover capabilities, Oracle implemented IB automatic path migration at the SDP level to avoid unnecessary failover operations at the middleware level. Security, for example spoof-protection. Another feature that is less important for traditional users of InfiniBand, but very important for enterprise customers. InfiniBand Partitioning and Quality-of-Service (QoS): One of the first questions we get from customers about Exalogic is: “How can we implement multi-tenancy?” The answer is to partition your IB network, which effectively creates many networks that work independently and that are protected at the lowest networking layer possible. In addition to that, QoS allows administrators to prioritize traffic flow in multi-tenancy environments so they can keep their service levels where it matters most. Resilient IB Fabric Management: InfiniBand is a self-managing network, so a lot of the magic lies in coming up with the right topology and in teaching the subnet manager how to properly discover and manage the network. Oracle's Infiniband switches come with pre-integrated, highly available fabric management with seamless integration into Oracle Enterprise Manager Ops Center. In short: Oracle elevated the OFED InfiniBand stack into an enterprise-class networking infrastructure. Many years and multiple teams of manpower went into the above improvements - this is something you can only get from Oracle, because no other InfiniBand vendor can give you these features across the whole stack! Exabus: Because it's not About the Size of Your Network, it's How You Use it! So let's assume that you somehow were able to get your hands on an enterprise-class IB driver stack. Or maybe you don't care and are just happy with the standard OFED one? Anyway, the next step is to actually leverage that InfiniBand performance. Here are the choices: Use traditional TCP/IP on top of the InfiniBand stack, Develop your own integration between your middleware and the lower-level (but faster) InfiniBand protocols. While more bandwidth is always a good thing, it's actually the low latency that enables superior performance for your applications when running on any networking infrastructure: The lower the latency, the faster the response travels through the network and the more transactions you can close per second. The reason why InfiniBand is such a low latency technology is that it gets rid of most if not all of your traditional networking protocol stack: Data is literally beamed from one region of RAM in one server into another region of RAM in another server with no kernel/drivers/UDP/TCP or other networking stack overhead involved! Which makes option 1 a no-go: Adding TCP/IP on top of InfiniBand is like adding training wheels to your racing bike. It may be ok in the beginning and for development, but it's not quite the performance IB was meant to deliver. Which only leaves option 2: Integrating your middleware with fast, low-level InfiniBand protocols. And this is what Exalogic's "Exabus" technology is all about. Here are a few Exabus features that help applications leverage the performance of InfiniBand in Exalogic: RDMA and SDP integration at the JDBC driver level (SDP), for Oracle Weblogic (SDP), Oracle Coherence (RDMA), Oracle Tuxedo (RDMA) and the new Oracle Traffic Director (RDMA) on Exalogic. Using these protocols, middleware can communicate a lot faster with each other and the Oracle database than by using standard networking protocols, Seamless Integration of Ethernet over InfiniBand from Exalogic's Gateway switches into the OS, Oracle Weblogic optimizations for handling massive amounts of parallel transactions. Because if you have an 8-lane Autobahn, you also need to improve your ramps so you can feed it with many cars in parallel. Integration of Weblogic with Oracle Exadata for faster performance, optimized session management and failover. As you see, “Exabus” is Oracle's word for describing all the InfiniBand enhancements Oracle put into Exalogic: OFED stack enhancements, protocols for faster IB access, and InfiniBand support and optimizations at the virtualization and middleware level. All working together to deliver the full potential of InfiniBand performance. Who else has 100% control over their middleware so they can develop their own low-level protocol integration with InfiniBand? Even if you take an open source approach, you're looking at years of development work to create, test and support a whole new networking technology in your middleware! The Extras: Less Hassle, More Productivity, Faster Time to Market And then there are the other advantages of Engineered Systems that are true for Exalogic the same as they are for every other Engineered System: One simple purchasing process: No headaches due to endless RFPs and no “Will X work with Y?” uncertainties. Everything has been engineered together: All kinds of bugs and problems have been already fixed at the design level that would have only manifested themselves after you have built the system from scratch. Everything is built, tested and integrated at the factory level . Less integration pain for you, faster time to market. Every Exalogic machine world-wide is identical to Oracle's own machines in the lab: Instant replication of any problems you may encounter, faster time to resolution. Simplified patching, management and operations. One throat to choke: Imagine finger-pointing hell for systems that have been put together using several different vendors. Oracle's Engineered Systems have a single phone number that customers can call to get their problems solved. For more business-centric values, read The Business Value of Engineered Systems. Conclusion: Buy Exalogic, or get ready for a 6-12 Month Science Project And here's the reason why it's not easy to "build your own Exalogic": There's a lot of work required to make such a system fly. In fact, anybody who is starting to "just put together a bunch of servers and an InfiniBand network" is really looking at a 6-12 month science project. And the outcome is likely to not be very enterprise-class. And it won't have Exalogic's performance either. Because building an Engineered System is literally rocket science: It takes a lot of time, effort, resources and many iterations of design/test/analyze/fix to build such a system. That's why InfiniBand has been reserved for HPC scientists for such a long time. And only Oracle can bring the power of InfiniBand in an enterprise-class, ready-to use, pre-integrated version to customers, without the develop/integrate/support pain. For more details, check the new Exalogic overview white paper which was updated only recently. P.S.: Thanks to my colleagues Ola, Paul, Don and Andy for helping me put together this article! var flattr_uid = '26528'; var flattr_tle = 'How to Avoid Your Next 12-Month Science Project'; var flattr_dsc = 'While most customers immediately understand how the magic of Oracle's Hybrid Columnar Compression, intelligent storage servers and flash memory make Exadata uniquely powerful against home-grown database systems, some people think that Exalogic is nothing more than a bunch of x86 servers, a storage appliance and an InfiniBand (IB) network, built into a single rack.After all, isn't this exactly what the High Performance Computing (HPC) world has been doing for decades?On the surface, this may be true. And some people tried exactly that: They tried to put together their own version of Exalogic, but then they discover there's a lot more to building a system than buying hardware and assembling it together. IT is not Ikea.Why is that so? Could it be there's more going on behind the scenes than merely putting together a bunch of servers, a storage array and an InfiniBand network into a rack? Let's explore some of the special sauce that makes Exalogic unique and un-copyable, so you can save yourself from your next 6- to 12-month science project that distracts you from doing real work that adds value to your company.'; var flattr_tag = 'Engineered Systems,Engineered Systems,Infiniband,Integration,latency,Oracle,performance'; var flattr_cat = 'text'; var flattr_url = 'http://constantin.glez.de/blog/2012/04/how-avoid-your-next-12-month-science-project'; var flattr_lng = 'en_GB'

  Read the article

• TCP RST right after FIN/ACK

  - by Nitzan Shaked

  I am having the weirdest issue: I have a web server which sometimes, only on very specific requests, will send a RST to the client after having sent the FIN datagram. First, a description of the setup: The server runs on an Ubuntu 12.04.1 LTS, which itself is a VM guest inside a Win7 x64 host, in bridged mode. ufw is disabled on the host The client runs on a iOS simulator, which runs on OS X Mountain Lion, which is a VM guest (hackintosh) inside a Win7 x64 host, in bridged mode. Both client and server are on the same LAN, one is connected to the home router via an Ethernet cable, and then other thru WiFi. I happened to glimpse over the server's http logs and found that the client sometimes issuing multiple subsequent identical requests. Further investigation led me to discover that this happens when the server sends a RST, and that the client is simply re-trying. I am attaching several tcpdump's: Good1 is the server-side tcpdump of a good session ("good" meaning no RST was generated). Good3 is another sever-side tcpdump of a good session. (The difference between Good1 and Good3 is the order in which ACK's were sent from the server to the client, ACK'ing the client's request. The client's request arives in 2 segements (specifically: one for the http headers, and another for a body containing an empty json object, "{}"). In Good1, the server ACK's both request segments, using 2 ACK segments, after the second request has arrived. In Good3, the server ACK's each request segment with an ACK segment as soon as the request segment arrives. Not that it should make a difference.) Bad1 is a dump, both client- and server-side, of a bad session. Bad2 is another bad session, this time server-side only. Note that in all "bad" sessions, the server ACK's each request segments immediately after having received it. I've looked at a few other bad sessions, and the situation is the same in all of them. But this is also the behavior in "Good3", so I don't see how that observation helps me, of for that matter why it should matter. I can't find any difference between good and bad sessions, or at least one that I think should matter. My question is: why are those RST's being generated? Or at least: how do I go about debugging this, or providing more info here that'll help? Edit 2 new facts that I have learned: Section 4.2.2.13 of the RFC (1122) (and Wikipedia, in the article "TCP", under "Connection Termination") says that a TCP application on one host may close the connection before it has read all of the data in its socket buffer, and in such a case the TCP on the host will sent a RST to the other side, to let it know that not all the data it has sent has been read. I'm not sure I completely understand this, since closing my side of the connection still allows me to read, no? It also means that I can't write any more. I am not sure this is relevant, though, since I see a RST after FIN. There are multiple complaints of this happening with wsgiref (Python's dev-mode HTTP server), which is exactly what I'm using. I'll keep updating as I find out more. Thanks! ~~~~~~~~~~~~~~~~~~~~ Good1 -- Server Side ~~~~~~~~~~~~~~~~~~~~ 13:28:02.308319 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [S], seq 94268074, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 943308864 ecr 0,sackOK,eol], length 0 13:28:02.308336 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [S.], seq 1726304574, ack 94268075, win 14480, options [mss 1460,sackOK,TS val 326480982 ecr 943308864,nop,wscale 3], length 0 13:28:02.309750 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [.], ack 1, win 8235, options [nop,nop,TS val 943308865 ecr 326480982], length 0 13:28:02.310744 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [P.], seq 1:351, ack 1, win 8235, options [nop,nop,TS val 943308865 ecr 326480982], length 350 13:28:02.310766 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [P.], seq 351:353, ack 1, win 8235, options [nop,nop,TS val 943308865 ecr 326480982], length 2 13:28:02.310841 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [.], ack 351, win 1944, options [nop,nop,TS val 326480983 ecr 943308865], length 0 13:28:02.310918 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [.], ack 353, win 1944, options [nop,nop,TS val 326480983 ecr 943308865], length 0 13:28:02.315931 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [P.], seq 1:18, ack 353, win 1944, options [nop,nop,TS val 326480984 ecr 943308865], length 17 13:28:02.316107 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [FP.], seq 18:684, ack 353, win 1944, options [nop,nop,TS val 326480984 ecr 943308865], length 666 13:28:02.317651 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [.], ack 18, win 8234, options [nop,nop,TS val 943308872 ecr 326480984], length 0 13:28:02.318288 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [.], ack 685, win 8192, options [nop,nop,TS val 943308872 ecr 326480984], length 0 13:28:02.318640 IP 192.168.1.51.51479 > 192.168.1.132.5000: Flags [F.], seq 353, ack 685, win 8192, options [nop,nop,TS val 943308872 ecr 326480984], length 0 13:28:02.318651 IP 192.168.1.132.5000 > 192.168.1.51.51479: Flags [.], ack 354, win 1944, options [nop,nop,TS val 326480985 ecr 943308872], length 0 ~~~~~~~~~~~~~~~~~~~~ Good3 -- Server Side ~~~~~~~~~~~~~~~~~~~~ 13:28:03.311143 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [S], seq 1982901126, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 943309853 ecr 0,sackOK,eol], length 0 13:28:03.311155 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [S.], seq 2245063571, ack 1982901127, win 14480, options [mss 1460,sackOK,TS val 326481233 ecr 943309853,nop,wscale 3], length 0 13:28:03.312671 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [.], ack 1, win 8235, options [nop,nop,TS val 943309854 ecr 326481233], length 0 13:28:03.313330 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [P.], seq 1:351, ack 1, win 8235, options [nop,nop,TS val 943309855 ecr 326481233], length 350 13:28:03.313337 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [.], ack 351, win 1944, options [nop,nop,TS val 326481234 ecr 943309855], length 0 13:28:03.313342 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [P.], seq 351:353, ack 1, win 8235, options [nop,nop,TS val 943309855 ecr 326481233], length 2 13:28:03.313346 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [.], ack 353, win 1944, options [nop,nop,TS val 326481234 ecr 943309855], length 0 13:28:03.327942 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [P.], seq 1:18, ack 353, win 1944, options [nop,nop,TS val 326481237 ecr 943309855], length 17 13:28:03.328253 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [FP.], seq 18:684, ack 353, win 1944, options [nop,nop,TS val 326481237 ecr 943309855], length 666 13:28:03.329076 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [.], ack 18, win 8234, options [nop,nop,TS val 943309868 ecr 326481237], length 0 13:28:03.329688 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [.], ack 685, win 8192, options [nop,nop,TS val 943309868 ecr 326481237], length 0 13:28:03.330361 IP 192.168.1.51.51486 > 192.168.1.132.5000: Flags [F.], seq 353, ack 685, win 8192, options [nop,nop,TS val 943309869 ecr 326481237], length 0 13:28:03.330370 IP 192.168.1.132.5000 > 192.168.1.51.51486: Flags [.], ack 354, win 1944, options [nop,nop,TS val 326481238 ecr 943309869], length 0 ~~~~~~~~~~~~~~~~~~~~ Bad1 -- Server Side ~~~~~~~~~~~~~~~~~~~~ 13:28:01.311876 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [S], seq 920400580, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 943307883 ecr 0,sackOK,eol], length 0 13:28:01.311896 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [S.], seq 3103085782, ack 920400581, win 14480, options [mss 1460,sackOK,TS val 326480733 ecr 943307883,nop,wscale 3], length 0 13:28:01.313509 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 1, win 8235, options [nop,nop,TS val 943307884 ecr 326480733], length 0 13:28:01.315614 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [P.], seq 1:351, ack 1, win 8235, options [nop,nop,TS val 943307886 ecr 326480733], length 350 13:28:01.315727 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [.], ack 351, win 1944, options [nop,nop,TS val 326480734 ecr 943307886], length 0 13:28:01.316229 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [P.], seq 351:353, ack 1, win 8235, options [nop,nop,TS val 943307886 ecr 326480733], length 2 13:28:01.316242 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [.], ack 353, win 1944, options [nop,nop,TS val 326480734 ecr 943307886], length 0 13:28:01.321019 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [P.], seq 1:18, ack 353, win 1944, options [nop,nop,TS val 326480735 ecr 943307886], length 17 13:28:01.321294 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [FP.], seq 18:684, ack 353, win 1944, options [nop,nop,TS val 326480736 ecr 943307886], length 666 13:28:01.321386 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R.], seq 685, ack 353, win 1944, options [nop,nop,TS val 326480736 ecr 943307886], length 0 13:28:01.322727 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 18, win 8234, options [nop,nop,TS val 943307891 ecr 326480735], length 0 13:28:01.322733 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R], seq 3103085800, win 0, length 0 13:28:01.323221 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 685, win 8192, options [nop,nop,TS val 943307892 ecr 326480736], length 0 13:28:01.323231 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R], seq 3103086467, win 0, length 0 ~~~~~~~~~~~~~~~~~~~~ Bad1 -- Client Side ~~~~~~~~~~~~~~~~~~~~ 13:28:11.374654 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [S], seq 920400580, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 943307883 ecr 0,sackOK,eol], length 0 13:28:11.375764 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [S.], seq 3103085782, ack 920400581, win 14480, options [mss 1460,sackOK,TS val 326480733 ecr 943307883,nop,wscale 3], length 0 13:28:11.376352 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 1, win 8235, options [nop,nop,TS val 943307884 ecr 326480733], length 0 13:28:11.378252 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [P.], seq 1:351, ack 1, win 8235, options [nop,nop,TS val 943307886 ecr 326480733], length 350 13:28:11.379027 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [P.], seq 351:353, ack 1, win 8235, options [nop,nop,TS val 943307886 ecr 326480733], length 2 13:28:11.379732 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [.], ack 351, win 1944, options [nop,nop,TS val 326480734 ecr 943307886], length 0 13:28:11.380592 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [.], ack 353, win 1944, options [nop,nop,TS val 326480734 ecr 943307886], length 0 13:28:11.384968 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [P.], seq 1:18, ack 353, win 1944, options [nop,nop,TS val 326480735 ecr 943307886], length 17 13:28:11.385044 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 18, win 8234, options [nop,nop,TS val 943307891 ecr 326480735], length 0 13:28:11.385586 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [FP.], seq 18:684, ack 353, win 1944, options [nop,nop,TS val 326480736 ecr 943307886], length 666 13:28:11.385743 IP 192.168.1.51.51472 > 192.168.1.132.5000: Flags [.], ack 685, win 8192, options [nop,nop,TS val 943307892 ecr 326480736], length 0 13:28:11.385966 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R.], seq 685, ack 353, win 1944, options [nop,nop,TS val 326480736 ecr 943307886], length 0 13:28:11.387343 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R], seq 3103085800, win 0, length 0 13:28:11.387344 IP 192.168.1.132.5000 > 192.168.1.51.51472: Flags [R], seq 3103086467, win 0, length 0 ~~~~~~~~~~~~~~~~~~~~ Bad2 -- Server Side ~~~~~~~~~~~~~~~~~~~~ 13:28:01.319185 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [S], seq 1631526992, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 943307889 ecr 0,sackOK,eol], length 0 13:28:01.319197 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [S.], seq 2524685719, ack 1631526993, win 14480, options [mss 1460,sackOK,TS val 326480735 ecr 943307889,nop,wscale 3], length 0 13:28:01.320692 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [.], ack 1, win 8235, options [nop,nop,TS val 943307890 ecr 326480735], length 0 13:28:01.322219 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [P.], seq 1:351, ack 1, win 8235, options [nop,nop,TS val 943307890 ecr 326480735], length 350 13:28:01.322336 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [.], ack 351, win 1944, options [nop,nop,TS val 326480736 ecr 943307890], length 0 13:28:01.322689 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [P.], seq 351:353, ack 1, win 8235, options [nop,nop,TS val 943307890 ecr 326480735], length 2 13:28:01.322700 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [.], ack 353, win 1944, options [nop,nop,TS val 326480736 ecr 943307890], length 0 13:28:01.326307 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [P.], seq 1:18, ack 353, win 1944, options [nop,nop,TS val 326480737 ecr 943307890], length 17 13:28:01.326614 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [FP.], seq 18:684, ack 353, win 1944, options [nop,nop,TS val 326480737 ecr 943307890], length 666 13:28:01.326710 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [R.], seq 685, ack 353, win 1944, options [nop,nop,TS val 326480737 ecr 943307890], length 0 13:28:01.328499 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [.], ack 18, win 8234, options [nop,nop,TS val 943307896 ecr 326480737], length 0 13:28:01.328509 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [R], seq 2524685737, win 0, length 0 13:28:01.328514 IP 192.168.1.51.51473 > 192.168.1.132.5000: Flags [.], ack 685, win 8192, options [nop,nop,TS val 943307896 ecr 326480737], length 0 13:28:01.328517 IP 192.168.1.132.5000 > 192.168.1.51.51473: Flags [R], seq 2524686404, win 0, length 0

  Read the article

< Previous Page | 1 2 3