Search Results

Search found 9519 results on 381 pages for 'bulk import'.

Page 38/381 | < Previous Page | 34 35 36 37 38 39 40 41 42 43 44 45  | Next Page >

  • How to finish a broken data upload to the production Google App Engine server?

    - by WooYek
    I was uploading the data to App Engine (not dev server) through loader class and remote api, and I hit the quota in the middle of a CSV file. Based on logs and progress sqllite db, how can I select remaining portion of data to be uploaded? Going through tens of records to determine which was and which was not transfered, is not appealing task, so I look for some way to limit the number of record I need to check. Here's relevant (IMO) log portion, how to interpret work item numbers? [DEBUG 2010-03-30 03:22:51,757 bulkloader.py] [Thread-2] [1041-1050] Transferred 10 entities in 3.9 seconds [DEBUG 2010-03-30 03:22:51,757 adaptive_thread_pool.py] [Thread-2] Got work item [1071-1080] <cut> [DEBUG 2010-03-30 03:23:09,194 bulkloader.py] [Thread-1] [1141-1150] Transferred 10 entities in 4.6 seconds [DEBUG 2010-03-30 03:23:09,194 adaptive_thread_pool.py] [Thread-1] Got work item [1161-1170] <cut> [DEBUG 2010-03-30 03:23:09,226 bulkloader.py] [Thread-3] [1151-1160] Transferred 10 entities in 4.2 seconds [DEBUG 2010-03-30 03:23:09,226 adaptive_thread_pool.py] [Thread-3] Got work item [1171-1180] [ERROR 2010-03-30 03:23:10,174 bulkloader.py] Retrying on non-fatal HTTP error: 503 Service Unavailable

    Read the article

  • Exporting query results to a file on the fly

    - by ercan
    Hi all, I need to export the results of a query to a csv file in an FTP folder. Is it possible to achieve this within a stored procedure? If yes, comes yet another constraint: can I achieve this without sysadmin privileges, aka without using xp_cmdshell + BCP utility? If no to 2., does the caller have to have sysadmin privileges or would it suffice if the SP owner has sysadmin privileges? Here are some more details to the problem: The SP must export and transfer the file on the fly and raise error if something went wrong. The caller must get a response immediately, i.e. in case of no error, he can assume that the results are successfully transferred to the folder. Therefore, a DTS/SSIS job that runs every N minutes is not an option. I know the problem smells like I will have to do this at application level, but I would be more than happy if all those stuff could be done from T-SQL.

    Read the article

  • Getting past Salesforce trigger governors

    - by Jake
    I'm trying to write an "after update" trigger that does a batch update on all child records of the record that has just been updated. This needs to be able to handle 15k+ child records at a time. Unfortunately, the limit appears to be 100, which is so far below my needs it's not even close to acceptable. I haven't tried splitting the records into batches of 100 each, since this will still put me at a cap of 10k updates per trigger execution. (Maybe I could just daisy-chain triggers together? ugh.) Does anyone know what series of hoops I can jump through to overcome yet another ridiculously short-sighted limitation by this awful development "platform"?

    Read the article

  • Can you automatically create a mysqldump file that doesn't enforce foreign key constraints?

    - by Tai Squared
    When I run a mysqldump command on my database and then try to import it, it fails as it attempts to create the tables alphabetically, even though they may have a foreign key that references a table later in the file. There doesn't appear to be anything in the documentation and I've found answers like this that say to update the file after it's created to include: set FOREIGN_KEY_CHECKS = 0; ...original mysqldump file contents... set FOREIGN_KEY_CHECKS = 1; Is there no way to automatically set those lines or export the tables in the necessary order (without having to manually specify all table names as that can be tedious and error prone)? I could wrap those lines in a script, but was wondering if there is an easy way to ensure I can dump a file and then import it without manually updating it.

    Read the article

  • Importing ctype; embedding python in C++ application

    - by Drew
    I'm trying to embed python within a C++ based programming language (CCL: The compuatational control language, not that any of you have heard of it). Thus, I don't really have a "main" function to make calls from. I have made a test .cc program with a main, and when I compile it and run it, I am able to import my own python modules and system modules for use. When I embed my code in my CCL-based program and compile it (with g++), it seems I have most functionality, but I get the import error: ImportError: /usr/lib/python2.6/lib-dynload/_ctypes.so: undefined symbol: PyType_GenericNew Can someone explain this to me and how to go about solving it? It seems like I've linked the objects correctly. Thanks.

    Read the article

  • How to use OpenCV in Python?

    - by Roman
    I have just installed OpenCV on my Windows 7 machine. As a result I get a new directory: C:\OpenCV2.2\Python2.7\Lib\site-packages In this directory I have two files: cv.lib and cv.pyd. Then I try to use the opencv from Python. I do the following: import sys sys.path.append('C:\OpenCV2.2\Python2.7\Lib\site-packages') import cv As a result I get the following error message: File "<stdin>", line 1, in <module> ImportError: DLL load failed: The specified module could not be found. What am I doing wrong? ADDED As it was recommended here, I have copied content of C:\OpenCV2.0\Python2.6\Lib\site-packages to the C:\Python26\Lib\site-packages. It did not help.

    Read the article

  • Python module seeing a full list as empty in another module

    - by Nick
    I'm working on a pygame project and have the main engine layed out. The problem is I hit a bug that I just can not seem to figure out. What happens is one module can't read a variable from another module. It's not that the variable can't be read, it just sees an empty list instead of what it really is. Instead of posting the entire source code I reproduced the bug in two small snippets that hopefully a skillful python-ist can interpret in his\her head. Code: main.py (This is the file that gets run) import screen screens = [] #A stack for all the game screens def current_screen(): #return a reference to the current screen return screens[-1] def play(): print'play called' current_screen().update() if __name__=='__main__': screens.append(screen.Screen()) play() screen.py import main class Screen: def __init__(self): print'screen made' def update(self): print main.screens #Should have a reference to itself in there Thanks!

    Read the article

  • Bulk get of child entities on Google app engine?

    - by dfrankow
    On Google App Engine in Python, I have a Visit entity that has a parent of Patient. A Patient may have multiple visits. I need to set the most_recent_visit (and some auxiliary visit data) somewhere for later querying, probably in another child entity that Brett Slatkin might call a "relationship index entity." I wish to do so in a bulk style as follows: 1. get 100 Patient keys 2. get all Visits that have any of the Patients from 1 as an ancestor 3. go through the Visits and get the latest for each Patient Is there any way to perform step 2 in a bulk get? I know there is a way to bulk get entities from a list of keys, and there is a way to match a single entity by its ancestor.

    Read the article

  • Faking user interaction on CMS

    - by Leocer
    I'm working with a CMS and need to import data to it using typical html forms. The data itself is in csv files with one page per row. Such is the CMS that importing directly to db isn't possible due to the complexity of the design. It's pretty important that i "fake" usual user interaction because the CMS does a lot of background work that's crucial for the import. Basically, for each row in the csv file, I need to copy a csv column to a html textfield, or select a checkbox, or click a certain button. One major issue is mapping the data in the csv to actions in the CMS. So if one column contains the string 'foobar' is really means "set the firstName dropdown widget to 'foobar'". Is there a tool to automate this? I´ve been looking at AutoHotKey, Selendium, Web-Harvester and many other tools but I'm not convinced they are the correct tools. The main problem is being able to interact with the html pages in a easy way.

    Read the article

  • Automating site interaction

    - by Leocer
    I'm working with a CMS and need to import data to it using typical html forms. The data itself is in csv files with one page per row. Such is the CMS that importing directly to db isn't possible due to the complexity of the design. It's pretty important that i "fake" usual user interaction because the CMS does a lot of background work that's crucial for the import. Basically, for each row in the csv file, I need to copy a csv column to a html textfield, or select a checkbox, or click a certain button. One major issue is mapping the data in the csv to actions in the CMS. So if one column contains the string 'foobar' is really means "set the firstName dropdown widget to 'foobar'". Is there a tool to automate this? I´ve been looking at AutoHotKey, Selendium, Web-Harvester and many other tools but I'm not convinced they are the correct tools. The main problem is being able to interact with the html pages in a easy way.

    Read the article

  • Scala importing a file in all files of a package

    - by Core_Dumped
    I need to use an implicit ordering that has been defined in an object in a file abc in the following way: object abc{ implicit def localTimeOrdering: Ordering[LocalDate] = Ordering.fromLessThan(_.isBefore(_)) } So, I make a package object xyz inside a file 'package.scala' that in turn is in the package 'xyz' that has files in which I need the implicit ordering to be applicable. I write something like this: package object xyz{ import abc._ } It does not seem to work. If I manually write the implicit definition statement inside the package object, it works perfectly. What is the correct way to import the object (abc) such that all of its objects/classes/definitions can be used in my entire package 'xyz' ?

    Read the article

  • User Controls Importing Problem (asp.net)(vb)

    - by Phil
    I have this class that contains vars for db connection; Public Shared s As String Public Shared con As String = WebConfigurationManager.ConnectionStrings("foo").ToString() Public Shared c As New SqlConnection(con) Public Shared x As New SqlCommand(s, c) I import this to my page like this; Imports DBVars I'm then able to access these vars from my page. But if I try to import them into a user control the variables are not available. Am I making an error or is this expected? Thanks.

    Read the article

  • How to create the automatic mass form submitter (javascript-ajax script) to be used on the 3rd part

    - by Daniel
    I need a script that can handle the following tasks. Take user data from my database and fill in and submit / post data to forms located on third part websites.: So I want to know if is it hard to create or do somebody knows if does exists some script for mass form submissions in PHP -Javascript-Ajax ? I run Dancers & Hostess & Model jobs website, I would like to find some script which allows the girls automaticly submit to hundreds websites forms (other 3rd part model agencies) with their similar model application form info on my website previously specified, 1).Firstly the girls will fill out my agency portfolio very detailed form , like this i will get all the model personal info from them , 2) Secondly i would like to allow for example models to submit to 100 and more other model agencies forms (I will find those websites before, and I will get their field names = values and thanks to some script would like to connect them with every girl data already created in my website to submit . I would like to implement it to my wordpress website where the girls has their portfolios instead of my pages . I would like to offer this service especially to models , it should work like some directory submitters , The script knows names - values and fill it out itself, but I want it online - browser side, where the girls should only fill out captcha if there is and click the button "submit".After succesful submit it should offer other form to submit. I would be very happy if you know the answer or if you can redirect me to some article

    Read the article

  • Load python module not from a file

    - by user575061
    Hello, I've got some python code in a library that attempts to load a simple value from a module that will exist for the applications that use this library from somemodule import simplevalue Normally, the application that uses the library will have the module file and everything works fine. However, in the unit tests for this library the module does not exist. I know that I can create a temporary file and add that file to my path at runtime, but I was curious if there is a way in python to load something in to memory that would allow the above import to work. This is more of a curiosity, saying "add the module to your test path" is not helpful :P

    Read the article

  • Xcode can't find imported header files

    - by solerous
    I've used other IDE's in the past but am new to Xcode. I'm trying to bring in a bunch of C code from an open source project. I've imported them into a new Group and the .c files all show up under Implementation Files and the full list of files shows up in the Groups/Files group as well as my project directory in the finder. When I try to include or import one of these header files, code completion even works so I know Xcode is seeing them. But then when I go to build it says "no such file or directory". How can I get these files to import into my code?

    Read the article

  • Importing data from many excel workbooks and sheets into a single workbook/table

    - by Max Rusalen
    Hi, I have 54 excel files with three sheets each, each sheet has a different amount of data entries but they are set out in a identical format, and I need to import the data from those sheets into a single workbook using VBA. Is there any way I can program it so I can build the loops to import the data, but without having to write in each workbook name for each loop/sheet? I think I can use the call function, but I don't know how to make the loop codes independent of the workbook name they apply to. Thank you so much in advance, Millie

    Read the article

  • Using Oracle Proxy Authentication with JPA (eclipselink-Style)

    - by olaf.heimburger
    Security is a very intriguing topic. You will find it everywhere and you need to implement it everywhere. Yes, you need. Unfortunately, one can easily forget it while implementing the last mile. The Last Mile In a multi-tier application it is a common practice to use connection pools between the business layer and the database layer. Connection pools are quite useful to speed database connection creation and to split the load. Another very common practice is to use a specific, often called technical, user to connect to the database. This user has authentication and authorization rules that apply to all application users. Imagine you've put every effort to define roles for different types of users that use your application. These roles are necessary to differentiate between normal users, premium users, and administrators (I bet you will find or already have more roles in your application). While these user roles are pretty well used within your application, once the flow of execution enters the database everything is gone. Each and every user just has one role and is the same database user. Issues? What Issues? As long as things go well, this is not a real issue. However, things do not go well all the time. Once your application becomes famous performance decreases in certain situations or, more importantly, current and upcoming regulations and laws require that your application must be able to apply different security measures on a per user role basis at every stage of your application. If you only have a bunch of users with the same name and role you are not able to find the application usage profile that causes the performance issue, or which user has accessed data that he/she is not allowed to. Another thread to your role concept is that databases tend to be used by different applications and tools. These tools can be developer tools like SQL*Plus, SQL Developer, etc. or end user applications like BI Publisher, Oracle Forms and so on. These tools have no idea of your applications role concept and access the database the way they think is appropriate. A big oversight for your perfect role model and a big nightmare for your Chief Security Officer. Speaking of the CSO, brings up another issue: Password management. Once your technical user account is compromised, every user is able to do things that he/she is not expected to do from the design of your application. Counter Measures In the Oracle world a common counter measure is to use Virtual Private Database (VPD). This restricts the values a database user can see to the allowed minimum. However, it doesn't help in regard of a connection pool user, because this one is still not the real user. Oracle Proxy Authentication Another feature of the Oracle database is Proxy Authentication. First introduced with version 9i it is a quite useful feature for nearly every situation. The main idea behind Proxy Authentication is, to create a crippled database user who has only connect rights. Even if this user is compromised the risks are well understood and fairly limited. This user can be used in every situation in which you need to connect to the database, no matter which tool or application (see above) you use.The proxy user is perfect for multi-tier connection pools. CREATE USER app_user IDENTIFIED BY abcd1234; GRANT CREATE SESSION TO app_user; But what if you need to access real data? Well, this is the primary use case, isn't it? Now is the time to bring the application's role concept into play. You define database roles that define the grants for your identified user groups. Once you have these groups you grant access through the proxy user with the application role to the specific user. CREATE ROLE app_role_a; GRANT app_role_a TO scott; ALTER USER scott GRANT CONNECT THROUGH app_user WITH ROLE app_role_a; Now, hr has permission to connect to the database through the proxy user. Through the role you can restrict the hr's rights the are needed for the application only. If hr connects to the database directly all assigned role and permissions apply. Testing the Setup To test the setup you can use SQL*Plus and connect to your database: $ sqlplus app_user[hr]/abcd1234 Java Persistence API The Java Persistence API (JPA) is a fairly easy means to build applications that retrieve data from the database and put it into Java objects. You use plain old Java objects (POJOs) and mixin some Java annotations that define how the attributes of the object are used for storing data from the database into the Java object. Here is a sample for objects from the HR sample schema EMPLOYEES table. When using Java annotations you only specify what can not be deduced from the code. If your Java class name is Employee but the table name is EMPLOYEES, you need to specify the table name, otherwise it will fail. package demo.proxy.ejb; import java.io.Serializable; import java.sql.Timestamp; import java.util.List; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.Id; import javax.persistence.JoinColumn; import javax.persistence.ManyToOne; import javax.persistence.NamedQueries; import javax.persistence.NamedQuery; import javax.persistence.OneToMany; import javax.persistence.Table; @Entity @NamedQueries({ @NamedQuery(name = "Employee.findAll", query = "select o from Employee o") }) @Table(name = "EMPLOYEES") public class Employee implements Serializable { @Column(name="COMMISSION_PCT") private Double commissionPct; @Column(name="DEPARTMENT_ID") private Long departmentId; @Column(nullable = false, unique = true, length = 25) private String email; @Id @Column(name="EMPLOYEE_ID", nullable = false) private Long employeeId; @Column(name="FIRST_NAME", length = 20) private String firstName; @Column(name="HIRE_DATE", nullable = false) private Timestamp hireDate; @Column(name="JOB_ID", nullable = false, length = 10) private String jobId; @Column(name="LAST_NAME", nullable = false, length = 25) private String lastName; @Column(name="PHONE_NUMBER", length = 20) private String phoneNumber; private Double salary; @ManyToOne @JoinColumn(name = "MANAGER_ID") private Employee employee; @OneToMany(mappedBy = "employee") private List employeeList; public Employee() { } public Employee(Double commissionPct, Long departmentId, String email, Long employeeId, String firstName, Timestamp hireDate, String jobId, String lastName, Employee employee, String phoneNumber, Double salary) { this.commissionPct = commissionPct; this.departmentId = departmentId; this.email = email; this.employeeId = employeeId; this.firstName = firstName; this.hireDate = hireDate; this.jobId = jobId; this.lastName = lastName; this.employee = employee; this.phoneNumber = phoneNumber; this.salary = salary; } public Double getCommissionPct() { return commissionPct; } public void setCommissionPct(Double commissionPct) { this.commissionPct = commissionPct; } public Long getDepartmentId() { return departmentId; } public void setDepartmentId(Long departmentId) { this.departmentId = departmentId; } public String getEmail() { return email; } public void setEmail(String email) { this.email = email; } public Long getEmployeeId() { return employeeId; } public void setEmployeeId(Long employeeId) { this.employeeId = employeeId; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public Timestamp getHireDate() { return hireDate; } public void setHireDate(Timestamp hireDate) { this.hireDate = hireDate; } public String getJobId() { return jobId; } public void setJobId(String jobId) { this.jobId = jobId; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getPhoneNumber() { return phoneNumber; } public void setPhoneNumber(String phoneNumber) { this.phoneNumber = phoneNumber; } public Double getSalary() { return salary; } public void setSalary(Double salary) { this.salary = salary; } public Employee getEmployee() { return employee; } public void setEmployee(Employee employee) { this.employee = employee; } public List getEmployeeList() { return employeeList; } public void setEmployeeList(List employeeList) { this.employeeList = employeeList; } public Employee addEmployee(Employee employee) { getEmployeeList().add(employee); employee.setEmployee(this); return employee; } public Employee removeEmployee(Employee employee) { getEmployeeList().remove(employee); employee.setEmployee(null); return employee; } } JPA could be used in standalone applications and Java EE containers. In both worlds you normally create a Facade to retrieve or store the values of the Entities to or from the database. The Facade does this via an EntityManager which will be injected by the Java EE container. Here is sample Facade Session Bean for a Java EE container. package demo.proxy.ejb; import java.util.HashMap; import java.util.List; import javax.ejb.Local; import javax.ejb.Remote; import javax.ejb.Stateless; import javax.persistence.EntityManager; import javax.persistence.PersistenceContext; import javax.persistence.Query; import javax.interceptor.AroundInvoke; import javax.interceptor.InvocationContext; import oracle.jdbc.driver.OracleConnection; import org.eclipse.persistence.config.EntityManagerProperties; import org.eclipse.persistence.internal.jpa.EntityManagerImpl; @Stateless(name = "DataFacade", mappedName = "ProxyUser-TestEJB-DataFacade") @Remote @Local public class DataFacadeBean implements DataFacade, DataFacadeLocal { @PersistenceContext(unitName = "TestEJB") private EntityManager em; private String username; public Object queryByRange(String jpqlStmt, int firstResult, int maxResults) { // setSessionUser(); Query query = em.createQuery(jpqlStmt); if (firstResult 0) { query = query.setFirstResult(firstResult); } if (maxResults 0) { query = query.setMaxResults(maxResults); } return query.getResultList(); } public Employee persistEmployee(Employee employee) { // setSessionUser(); em.persist(employee); return employee; } public Employee mergeEmployee(Employee employee) { // setSessionUser(); return em.merge(employee); } public void removeEmployee(Employee employee) { // setSessionUser(); employee = em.find(Employee.class, employee.getEmployeeId()); em.remove(employee); } /** select o from Employee o */ public List getEmployeeFindAll() { Query q = em.createNamedQuery("Employee.findAll"); return q.getResultList(); } Putting Both Together To use Proxy Authentication with JPA and within a Java EE container you have to take care of the additional requirements: Use an OCI JDBC driver Provide the user name that connects through the proxy user Use an OCI JDBC driver To use the OCI JDBC driver you need to set up your JDBC data source file to use the correct JDBC URL. hr jdbc:oracle:oci8:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521))(CONNECT_DATA=(SID=XE))) oracle.jdbc.OracleDriver user app_user 62C32F70E98297522AD97E15439FAC0E SQL SELECT 1 FROM DUAL jdbc/hrDS Application Additionally you need to make sure that the version of the shared libraries of the OCI driver match the version of the JDBC driver in your Java EE container or Java application and are within your PATH (on Windows) or LD_LIBRARY_PATH (on most Unix-based systems). Installing the Oracle Database Instance Client software works perfectly. Provide the user name that connects through the proxy user This part needs some modification of your application software and session facade. Session Facade Changes In the Session Facade we must ensure that every call that goes through the EntityManager must be prepared correctly and uniquely assigned to this session. The second is really important, as the EntityManager works with a connection pool and can not guarantee that we set the proxy user on the connection that will be used for the database activities. To avoid changing every method call of the Session Facade we provide a method to set the username of the user that connects through the proxy user. This method needs to be called by the Facade client bfore doing anything else. public void setUsername(String name) { username = name; } Next we provide a means to instruct the TopLink EntityManager Delegate to use Oracle Proxy Authentication. (I love small helper methods to hide the nitty-gritty details and avoid repeating myself.) private void setSessionUser() { setSessionUser(username); } private void setSessionUser(String user) { if (user != null && !user.isEmpty()) { EntityManagerImpl emDelegate = ((EntityManagerImpl)em.getDelegate()); emDelegate.setProperty(EntityManagerProperties.ORACLE_PROXY_TYPE, OracleConnection.PROXYTYPE_USER_NAME); emDelegate.setProperty(OracleConnection.PROXY_USER_NAME, user); emDelegate.setProperty(EntityManagerProperties.EXCLUSIVE_CONNECTION_MODE, "Always"); } } The final step is use the EJB 3.0 AroundInvoke interceptor. This interceptor will be called around every method invocation. We therefore check whether the Facade methods will be called or not. If so, we set the user for proxy authentication and the normal method flow continues. @AroundInvoke public Object proxyInterceptor(InvocationContext invocationCtx) throws Exception { if (invocationCtx.getTarget() instanceof DataFacadeBean) { setSessionUser(); } return invocationCtx.proceed(); } Benefits Using Oracle Proxy Authentification has a number of additional benefits appart from implementing the role model of your application: Fine grained access control for temporary users of the account, without compromising the original password. Enabling database auditing and logging. Better identification of performance bottlenecks. References Effective Oracle Database 10g Security by Design, David Knox TopLink Developer's Guide, Chapter 98

    Read the article

  • NoClassDefFoundError and Netty

    - by Dmytro Leonenko
    Hi. First to say I'm n00b in Java. I can understand most concepts but in my situation I want somebody to help me. I'm using JBoss Netty to handle simple http request and using MemCachedClient check existence of client ip in memcached. import org.jboss.netty.channel.ChannelHandler; import static org.jboss.netty.handler.codec.http.HttpHeaders.*; import static org.jboss.netty.handler.codec.http.HttpHeaders.Names.*; import static org.jboss.netty.handler.codec.http.HttpResponseStatus.*; import static org.jboss.netty.handler.codec.http.HttpVersion.*; import com.danga.MemCached.*; import java.util.List; import java.util.Map; import java.util.Map.Entry; import java.util.Set; import org.jboss.netty.buffer.ChannelBuffer; import org.jboss.netty.buffer.ChannelBuffers; import org.jboss.netty.channel.ChannelFuture; import org.jboss.netty.channel.ChannelFutureListener; import org.jboss.netty.channel.ChannelHandlerContext; import org.jboss.netty.channel.ExceptionEvent; import org.jboss.netty.channel.MessageEvent; import org.jboss.netty.channel.SimpleChannelUpstreamHandler; import org.jboss.netty.handler.codec.http.Cookie; import org.jboss.netty.handler.codec.http.CookieDecoder; import org.jboss.netty.handler.codec.http.CookieEncoder; import org.jboss.netty.handler.codec.http.DefaultHttpResponse; import org.jboss.netty.handler.codec.http.HttpChunk; import org.jboss.netty.handler.codec.http.HttpChunkTrailer; import org.jboss.netty.handler.codec.http.HttpRequest; import org.jboss.netty.handler.codec.http.HttpResponse; import org.jboss.netty.handler.codec.http.HttpResponseStatus; import org.jboss.netty.handler.codec.http.QueryStringDecoder; import org.jboss.netty.util.CharsetUtil; /** * @author <a href="http://www.jboss.org/netty/">The Netty Project</a> * @author Andy Taylor ([email protected]) * @author <a href="http://gleamynode.net/">Trustin Lee</a> * * @version $Rev: 2368 $, $Date: 2010-10-18 17:19:03 +0900 (Mon, 18 Oct 2010) $ */ @SuppressWarnings({"ALL"}) public class HttpRequestHandler extends SimpleChannelUpstreamHandler { private HttpRequest request; private boolean readingChunks; /** Buffer that stores the response content */ private final StringBuilder buf = new StringBuilder(); protected MemCachedClient mcc = new MemCachedClient(); private static SockIOPool poolInstance = null; static { // server list and weights String[] servers = { "lcalhost:11211" }; //Integer[] weights = { 3, 3, 2 }; Integer[] weights = {1}; // grab an instance of our connection pool SockIOPool pool = SockIOPool.getInstance(); // set the servers and the weights pool.setServers(servers); pool.setWeights(weights); // set some basic pool settings // 5 initial, 5 min, and 250 max conns // and set the max idle time for a conn // to 6 hours pool.setInitConn(5); pool.setMinConn(5); pool.setMaxConn(250); pool.setMaxIdle(21600000); //1000 * 60 * 60 * 6 // set the sleep for the maint thread // it will wake up every x seconds and // maintain the pool size pool.setMaintSleep(30); // set some TCP settings // disable nagle // set the read timeout to 3 secs // and don't set a connect timeout pool.setNagle(false); pool.setSocketTO(3000); pool.setSocketConnectTO(0); // initialize the connection pool pool.initialize(); // lets set some compression on for the client // compress anything larger than 64k //mcc.setCompressEnable(true); //mcc.setCompressThreshold(64 * 1024); } @Override public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) throws Exception { HttpRequest request = this.request = (HttpRequest) e.getMessage(); if(mcc.get(request.getHeader("X-Real-Ip")) != null) { HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK); response.setHeader("X-Accel-Redirect", request.getUri()); ctx.getChannel().write(response).addListener(ChannelFutureListener.CLOSE); } else { sendError(ctx, NOT_FOUND); } } private void writeResponse(MessageEvent e) { // Decide whether to close the connection or not. boolean keepAlive = isKeepAlive(request); // Build the response object. HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK); response.setContent(ChannelBuffers.copiedBuffer(buf.toString(), CharsetUtil.UTF_8)); response.setHeader(CONTENT_TYPE, "text/plain; charset=UTF-8"); if (keepAlive) { // Add 'Content-Length' header only for a keep-alive connection. response.setHeader(CONTENT_LENGTH, response.getContent().readableBytes()); } // Encode the cookie. String cookieString = request.getHeader(COOKIE); if (cookieString != null) { CookieDecoder cookieDecoder = new CookieDecoder(); Set<Cookie> cookies = cookieDecoder.decode(cookieString); if(!cookies.isEmpty()) { // Reset the cookies if necessary. CookieEncoder cookieEncoder = new CookieEncoder(true); for (Cookie cookie : cookies) { cookieEncoder.addCookie(cookie); } response.addHeader(SET_COOKIE, cookieEncoder.encode()); } } // Write the response. ChannelFuture future = e.getChannel().write(response); // Close the non-keep-alive connection after the write operation is done. if (!keepAlive) { future.addListener(ChannelFutureListener.CLOSE); } } @Override public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) throws Exception { e.getCause().printStackTrace(); e.getChannel().close(); } private void sendError(ChannelHandlerContext ctx, HttpResponseStatus status) { HttpResponse response = new DefaultHttpResponse(HTTP_1_1, status); response.setHeader(CONTENT_TYPE, "text/plain; charset=UTF-8"); response.setContent(ChannelBuffers.copiedBuffer( "Failure: " + status.toString() + "\r\n", CharsetUtil.UTF_8)); // Close the connection as soon as the error message is sent. ctx.getChannel().write(response).addListener(ChannelFutureListener.CLOSE); } } When I try to send request like http://127.0.0.1:8090/1/2/3 I'm getting java.lang.NoClassDefFoundError: com/danga/MemCached/MemCachedClient at httpClientValidator.server.HttpRequestHandler.<clinit>(HttpRequestHandler.java:66) I believe it's not related to classpath. May be it's related to context in which mcc doesn't exist. Any help appreciated EDIT: Original code http://docs.jboss.org/netty/3.2/xref/org/jboss/netty/example/http/snoop/package-summary.html I've modified some parts to fit my needs.

    Read the article

  • error initializing multiple configuration files

    - by lurscher
    Hi, during initialization startup on tomcat, the configurations are: 1) a webapp/WEB-INF/web.xml that imports yummy-servlet.xml in contextConfigLocation (although i'm aware that is not required since the servlet-name is yummy it will try to load yummy-servlet.xml by default) 2) a webapp/WEB-INF/yummy-servlet.xml that imports a spring/applicationContext-hibernate.xml file 3) a webapp/WEB-INF/spring/applicationContext-hibernate.xml that imports a applicationContext-dataSource.xml file 4) a webapp/WEB-INF/spring/applicationContext-dataSource.xml i'm getting errors about Failed to import bean definitions from relative location, but the stack trace is not very explicit about exactly what is the problem, i've been looking at these since yesterday and i really don't see any problem on the files my web.xml: <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5"> <display-name>hello-spring3-RC1</display-name> <context-param> <param-name>contextConfigLocation</param-name> <param-value>/WEB-INF/yummy-servlet.xml</param-value> </context-param> <!-- <context-param> <param-name>contextConfigLocation</param-name> <param-value>/WEB-INF/spring/applicationContext-hibernate.xml</param-value> </context-param> --> <!-- Location of the Log4J config file, for initialization and refresh checks. Applied by Log4jConfigListener. --> <context-param> <param-name>log4jConfigLocation</param-name> <param-value>classpath:log4j.properties</param-value> </context-param> <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> <servlet> <servlet-name>yummy</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>yummy</servlet-name> <url-pattern>*.html</url-pattern> </servlet-mapping> <welcome-file-list> <welcome-file>index.html</welcome-file> </welcome-file-list> </web-app> my yummy-servlet.xml: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd"> <import resource="spring/applicationContext-hibernate.xml"/> <context:component-scan base-package="com.mine.web.controllers"/> <bean id="jspViewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/> <property name="prefix" value="/WEB-INF/jsp/"/> <property name="suffix" value=".jsp"/> </bean> </beans> my applicationContext-hibernate.xml: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"> <!-- import the dataSource definition --> <import resource="applicationContext-dataSource.xml"/> <!-- Configurer that replaces ${...} placeholders with values from a properties file --> <!-- (in this case, Hibernate-related settings for the sessionFactory definition below) --> <context:property-placeholder location="classpath:jdbc.properties"/> <context:property-placeholder location="classpath:hibernate.properties"/> <!-- Hibernate SessionFactory --> <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean" p:dataSource-ref="dataSource" p:mappingResources="hello.hbm.xml"> <property name="hibernateProperties"> <props> <prop key="hibernate.dialect">${hibernate.dialect}</prop> <prop key="hibernate.show_sql">${hibernate.show_sql}</prop> <prop key="hibernate.generate_statistics">${hibernate.generate_statistics}</prop> </props> </property> <property name="eventListeners"> <map> <entry key="merge"> <bean class="org.springframework.orm.hibernate3.support.IdTransferringMergeEventListener"/> </entry> </map> </property> </bean> <bean id="hibernateTemplate" class="org.springframework.orm.hibernate3.HibernateTemplate"> <property name="sessionFactory" ref="sessionFactory" /> </bean> <!-- Transaction manager for a single Hibernate SessionFactory (alternative to JTA) --> <bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager" p:sessionFactory-ref="sessionFactory"/> <!-- ========================= BUSINESS OBJECT DEFINITIONS ========================= --> <!-- Activates various annotations to be detected in bean classes: Spring's @Required and @Autowired, as well as JSR 250's @Resource. --> <context:annotation-config/> <!-- Instruct Spring to perform declarative transaction management automatically on annotated classes. --> <tx:annotation-driven transaction-manager="transactionManager"/> <bean id="EntityManager" class="com.mine.persistence.hibernate.HibernateHelloWorldDao"/> </beans> and my applicationContext-dataSource.xml: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:context="http://www.springframework.org/schema/context" xmlns:jdbc="http://www.springframework.org/schema/jdbc" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd"> <!-- Configurer that replaces ${...} placeholders with values from a properties file --> <!-- (in this case, JDBC-related settings for the dataSource definition below) --> <context:property-placeholder location="classpath:jdbc.properties"/> <context:property-placeholder location="classpath:hibernate.properties"/> <!-- data source using apache common dbcp pool manager <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" p:driverClassName="${jdbc.driverClassName}" p:url="${jdbc.url}" p:username="${jdbc.username}" p:password="${jdbc.password}"/>--> <!-- c3p0 pool manager data source --> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close"> <property name="driverClass" value="${jdbc.driverClassName}"/> <property name="jdbcUrl" value="${jdbc.url}"/> <property name="user" value="${jdbc.username}"/> <property name="password" value="${jdbc.password}"/> <property name="initialPoolSize" value="${hibernate.c3p0.min_size}"/> <property name="minPoolSize" value="${hibernate.c3p0.min_size}"/> <property name="maxPoolSize" value="${jdbc.maxconn}"/> <property name="idleConnectionTestPeriod" value="150"/> <property name="acquireIncrement" value="1"/> <property name="maxStatements" value="0"/> <property name="numHelperThreads" value="5"/> </bean> </beans> and this is the stack trace: 2010-06-13 12:16:33,526 INFO [org.springframework.web.context.ContextLoader] - < Root WebApplicationContext: initialization started> 2010-06-13 12:16:33,707 INFO [org.springframework.web.context.support.XmlWebAppl icationContext] - <Refreshing Root WebApplicationContext: startup date [Sun Jun 13 12:16:33 GMT-05:00 2010]; root of context hierarchy> 2010-06-13 12:16:34,086 INFO [org.springframework.beans.factory.xml.XmlBeanDefin itionReader] - <Loading XML bean definitions from ServletContext resource [/WEB- INF/yummy-servlet.xml]> 2010-06-13 12:16:34,378 INFO [org.springframework.beans.factory.xml.XmlBeanDefin itionReader] - <Loading XML bean definitions from URL [jndi:/localhost/protoweb/ WEB-INF/spring/applicationContext-hibernate.xml]> 2010-06-13 12:16:34,473 INFO [org.springframework.beans.factory.xml.XmlBeanDefin itionReader] - <Loading XML bean definitions from URL [jndi:/localhost/protoweb/ WEB-INF/spring/applicationContext-dataSource.xml]> 2010-06-13 12:16:35,098 ERROR [org.springframework.web.context.ContextLoader] - <Context initialization failed> org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Failed to import bean definitions from relative location [spring/applicationContext-hibernate.xml] Offending resource: ServletContext resource [/WEB-INF/yummy-servlet.xml]; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Un expected exception parsing XML document from URL [jndi:/localhost/protoweb/WEB-INF/spring/applicationContext-hibernate.xml]; nested exception is java.lang.NoSuchMethodError: org.springframework.beans.MutablePropertyValues.add(Ljava/lang/Str ing;Ljava/lang/Object;)Lorg/springframework/beans/MutablePropertyValues; at org.springframework.beans.factory.parsing.FailFastProblemReporter.err or(FailFastProblemReporter.java:68) at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderC ontext.java:85) at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderC ontext.java:76) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.importBeanDefinitionResource(DefaultBeanDefinitionDocumentReader.java:197) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.parseDefaultElement(DefaultBeanDefinitionDocumentReader.java:146) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:131) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:91) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registe rBeanDefinitions(XmlBeanDefinitionReader.java:475) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadB eanDefinitions(XmlBeanDefinitionReader.java:372) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBea nDefinitions(XmlBeanDefinitionReader.java:316) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBea nDefinitions(XmlBeanDefinitionReader.java:284) at org.springframework.beans.factory.support.AbstractBeanDefinitionReade r.loadBeanDefinitions(AbstractBeanDefinitionReader.java:143) at org.springframework.beans.factory.support.AbstractBeanDefinitionReade r.loadBeanDefinitions(AbstractBeanDefinitionReader.java:178) at org.springframework.beans.factory.support.AbstractBeanDefinitionReade r.loadBeanDefinitions(AbstractBeanDefinitionReader.java:149) at org.springframework.web.context.support.XmlWebApplicationContext.load BeanDefinitions(XmlWebApplicationContext.java:125) at org.springframework.web.context.support.XmlWebApplicationContext.load BeanDefinitions(XmlWebApplicationContext.java:93) at org.springframework.context.support.AbstractRefreshableApplicationCon text.refreshBeanFactory(AbstractRefreshableApplicationContext.java:127) at org.springframework.context.support.AbstractApplicationContext.obtain FreshBeanFactory(AbstractApplicationContext.java:429) at org.springframework.context.support.AbstractApplicationContext.refres h(AbstractApplicationContext.java:356) at org.springframework.web.context.ContextLoader.createWebApplicationCon text(ContextLoader.java:270) at org.springframework.web.context.ContextLoader.initWebApplicationConte xt(ContextLoader.java:197) at org.springframework.web.context.ContextLoaderListener.contextInitiali zed(ContextLoaderListener.java:47) at org.apache.catalina.core.StandardContext.listenerStart(StandardContex t.java:3972) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4 467) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase .java:791) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:77 1) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:905) at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:740 ) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:500 ) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java :321) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Lifecycl eSupport.java:119) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) at org.apache.catalina.core.StandardHost.start(StandardHost.java:785) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443 ) at org.apache.catalina.core.StandardService.start(StandardService.java:5 19) at org.apache.catalina.core.StandardServer.start(StandardServer.java:710 ) at org.apache.catalina.startup.Catalina.start(Catalina.java:581) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Unexp ected exception parsing XML document from URL [jndi:/localhost/protoweb/WEB-INF/ spring/applicationContext-hibernate.xml]; nested exception is java.lang.NoSuchMe thodError: org.springframework.beans.MutablePropertyValues.add(Ljava/lang/String ;Ljava/lang/Object;)Lorg/springframework/beans/MutablePropertyValues; at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadB eanDefinitions(XmlBeanDefinitionReader.java:394) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBea nDefinitions(XmlBeanDefinitionReader.java:316) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBea nDefinitions(XmlBeanDefinitionReader.java:284) at org.springframework.beans.factory.support.AbstractBeanDefinitionReade r.loadBeanDefinitions(AbstractBeanDefinitionReader.java:143) at org.springframework.beans.factory.support.AbstractBeanDefinitionReade r.loadBeanDefinitions(AbstractBeanDefinitionReader.java:178) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.importBeanDefinitionResource(DefaultBeanDefinitionDocumentReader.java:187) ... 42 more Caused by: java.lang.NoSuchMethodError: org.springframework.beans.MutablePropert yValues.add(Ljava/lang/String;Ljava/lang/Object;)Lorg/springframework/beans/Muta blePropertyValues; at org.springframework.transaction.config.AnnotationDrivenBeanDefinition Parser.registerTransactionManager(AnnotationDrivenBeanDefinitionParser.java:95) at org.springframework.transaction.config.AnnotationDrivenBeanDefinition Parser.access$0(AnnotationDrivenBeanDefinitionParser.java:94) at org.springframework.transaction.config.AnnotationDrivenBeanDefinition Parser$AopAutoProxyConfigurer.configureAutoProxyCreator(AnnotationDrivenBeanDefi nitionParser.java:121) at org.springframework.transaction.config.AnnotationDrivenBeanDefinition Parser.parse(AnnotationDrivenBeanDefinitionParser.java:79) at org.springframework.beans.factory.xml.NamespaceHandlerSupport.parse(N amespaceHandlerSupport.java:72) at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.pa rseCustomElement(BeanDefinitionParserDelegate.java:1327) at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.pa rseCustomElement(BeanDefinitionParserDelegate.java:1317) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:134) at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentRe ader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:91) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registe rBeanDefinitions(XmlBeanDefinitionReader.java:475) at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadB eanDefinitions(XmlBeanDefinitionReader.java:372) ... 47 more 06/13/2010 12:16:35 PM org.apache.catalina.core.StandardContext start SEVERE: Error listenerStart

    Read the article

  • SQL Server 2005: Improving performance for thousands or Insert requests. logout-login time= 120ms.

    - by Rad
    Can somebody shed some lights on how SQL Server 2005 deals with may request issued by a client using ADO.NET 2.0. Below is the shortend output of SQL Trace. I can see that connection pooling is working (I believe there is only one connection being pooled). What is not clear to me is why we have so many sp_reset_connection calls i.e a series of: Audit Login, SQL:BatchStarting, RPC:Starting and Audit Logout for each loop in for loop below. I can see that there is constant switching between tempdb and master database which leads me to conclude that we lost the context when next connection is created by fetching it from the pool based on ConectionString argument. I can see that every 15ms I can get 100-200 login/logout per second (reported at the same time by Profiler). The after 15ms I have again a series fo 100-200 login/logout per second. I need clarification on how this might affect much complex insert queries in production environment. I use Enterprise Library 2006, the code is compiled with VS 2005 and it is a console application that parses a flat file with 10 of thousand of rows grouping parent-child rows, runs on an application server and runs 2 stored procedure on a remote SQL Server 2005 inserting a parent record, retrieves Identity value and using it calls the second stored procedure 1, 2 or multiple times (sometimes several thousands) inserting child records. The child table has close to 10 million records with 5-10 indexes some of them being covering non-clustered. There is a pretty complex Insert trigger that copies inserted detail record to an archive table. All in all I only have 7 inserts per second which means it can take 2-4 hours for 50 thousand records. When I run Profiler on the test server (that is almost equivalent with production server) I can see that there is about 120ms between Audit Logout and Audit Login trace entries which almost give me chance to insert about 8 records. So my question is if there is some way to improve inserting of records since the company loads 100 thousands of records and does daily planning and has SLA to fulfill client request coming as flat file orders and some big files 10 thousands have to be processed(imported quickly). 4 hours to import 60 thousands should be reduced to 30 minutes. I was thinking to use BatchSize of DataAdapter to send multiple stored procedure calls, SQL Bulk inserts to batch multiple inserts from DataReader or DataTable, SSIS fast load. But I don't know how to properly analyze re-indexing and stats population and maybe this has to take some time to finish. What is worse is that the company uses the biggest table for reporting and other online processing and indexes cannot be dropped. I manage transaction manually by setting a field to a value and do an transactional update changing that value to a new value that other applications are using to get committed rows. Please advise how to approach this problem. For now I am trying to have a staging tables with minimal logging in a separate database and no indexes and I will try to do batched (massive) parent child inserts. I believe Production DB has simple recovery model, but it could be full recovery. If DB user that is being used by my .NET console application has bulkadmin role does it mean its bulk inserts are minimally logged. I understand that when a table has clustered and many non-clustered indexes that inserts are still logged for each row. Connection pooling is working, but with many login/logouts. Why? for (int i = 1; i <= 10000; i++){ using (SqlConnection conn = new SqlConnection("server=(local);database=master;integrated security=sspi;")) {conn.Open(); using (SqlCommand cmd = conn.CreateCommand()){ cmd.CommandText = "use tempdb"; cmd.ExecuteNonQuery();}}} SQL Server Profiler trace: Audit Login master 2010-01-13 23:18:45.337 1 - Nonpooled SQL:BatchStarting use tempdb master 2010-01-13 23:18:45.337 RPC:Starting exec sp_reset_conn tempdb 2010-01-13 23:18:45.337 Audit Logout tempdb 2010-01-13 23:18:45.337 2 - Pooled Audit Login -- network protocol master 2010-01-13 23:18:45.383 2 - Pooled SQL:BatchStarting use tempdb master 2010-01-13 23:18:45.383 RPC:Starting exec sp_reset_conn tempdb 2010-01-13 23:18:45.383 Audit Logout tempdb 2010-01-13 23:18:45.383 2 - Pooled Audit Login -- network protocol master 2010-01-13 23:18:45.383 2 - Pooled SQL:BatchStarting use tempdb master 2010-01-13 23:18:45.383 RPC:Starting exec sp_reset_conn tempdb 2010-01-13 23:18:45.383 Audit Logout tempdb 2010-01-13 23:18:45.383 2 - Pooled

    Read the article

  • Importing Excel to SQLCE

    - by ohadsc
    I have a table in excel format (2007 but I can save as anything below that, naturally), and I have an SQL Compact Edition 3.5 SP1 Database table with corresponding columns. I simply want to populate the SQLCE table with the data from the excel file. The data consists of strings and integers only. I tried this utility to no avail, I also tried this SQL script but it won't work since BULK INSERT is not supported in SQLCE. I also found this Microsoft tutorial but I am basically clueless when it comes to SQL, providers and the like... Thanks !

    Read the article

  • Using DevIL with Xcode

    - by jak3t
    Hello everyone! I'm trying to use DevIL with XCode but I can't get it to work. I tried using the "standard" method by doing "configure make make install" and using #import <IL/il> but it doesn't work. It identifies the library but I still get some compiling errors: Ld build/Release/cg.app/Contents/MacOS/cg normal i386 cd "/Users/simaorfreitas/Documents/LESI/3º ano/CG/CGaula1" /Developer/usr/bin/g++-4.2 -arch i386 "-L/Users/simaorfreitas/Documents/LESI/3º ano/CG/CGaula1/build/Release" "-F/Users/simaorfreitas/Documents/LESI/3º ano/CG/CGaula1/build/Release" -filelist "/Users/simaorfreitas/Documents/LESI/3º ano/CG/CGaula1/build/CGaula1.build/Release/cg.build/Objects-normal/i386/cg.LinkFileList" -framework Foundation -framework AppKit -framework GLUT -framework OpenGL -framework Cocoa -framework AppKit -o "/Users/simaorfreitas/Documents/LESI/3º ano/CG/CGaula1/build/Release/cg.app/Contents/MacOS/cg" Undefined symbols: "_ilConvertImage", referenced from: terreno() in main.o "_ilGetInteger", referenced from: terreno() in main.o terreno() in main.o terreno() in main.o "_ilLoadImage", referenced from: terreno() in main.o "_ilBindImage", referenced from: terreno() in main.o "_ilGetData", referenced from: terreno() in main.o "_ilInit", referenced from: terreno() in main.o "_ilGenImages", referenced from: terreno() in main.o ld: symbol(s) not found collect2: ld returned 1 exit status` So i tried do it by using the macports libdevil install. It installed correctly but nothing seems to work. Do I need to change my #import declarations? Any suggestion to install DevIL on the Mac correctly and use it in a XCode Project? Thanks a lot!

    Read the article

  • Destroying descendants for dojox.layout.ContentPane

    - by Luigi
    Hi, I just want to clean all the content of my dojox pane before load another page. There is an attribute called cleanContent that seems not working. Anyway this is a jsp page I import: <head> <script type="text/javascript" src="./js/listMetadataClass.js"></script> <script type="text/javascript" src="./js/utility.js"></script> <link rel="stylesheet" type="text/css" href="../dojo/dijit/themes/tundra/tundra.css"> <link rel="stylesheet" type="text/css" href="../dojo/dojox/grid/resources/tundraGrid.css"> </head> <div dojoType="dijit.layout.ContentPane"> <h3>List of existing Metadata Class:</h3> <table id="gridNode" style="width:auto; height:300px;"></table> <div id="dialogNode"></div> </div> Anyway is this the correct way to import a page inside the dojox pane? Thanks Luigi

    Read the article

  • How to combine twill and python into one code that could be run on "Google App Engine"?

    - by brilliant
    Hello everybody!!! I have installed twill on my computer (having previously installed Python 2.5) and have been using it recently. Python is installed on disk C on my computer: C:\Python25 And the twill folder (“twill-0.9”) is located here: E:\tmp\twill-0.9 Here is a code that I’ve been using in twill: go “some website’s sign-in page URL” formvalue 2 userid “my login” formvalue 2 pass “my password” submit go “URL of some other page from that website” save_html result.txt This code helps me to log in to one website, in which I have an account, record the HTML code of some other page of that website (that I can access only after logging in), and store it in a file named “result.txt” (of course, before using this code I firstly need to replace “my login” with my real login, “my password” with my real password, “some website’s sign-in page URL” and “URL of some other page from that website” with real URLs of that website, and number 2 with the number of the form on that website that is used as a sign-in form on that website’s log-in page) This code I store in “test.twill” file that is located in my “twill-0.9” folder: E:\tmp\twill-0.9\test.twill I run this file from my command prompt: python twill-sh test.twill Now, I also have installed “Google App Engine SDK” from “Google App Engine” and have also been using it for awhile. For example, I’ve been using this code: import hashlib m = hashlib.md5() m.update("Nobody inspects") m.update(" the spammish repetition ") print m.hexdigest() This code helps me transform the phrase “Nobody inspects the spammish repetition” into md5 digest. Now, how can I put these two pieces of code together into one python script that I could run on “Google App Engine”? Let’s say, I want my code to log in to a website from “Google App Engine”, go to another page on that website, record its HTML code (that’s what my twill code does) and than transform this HTML code into its md5 digest (that’s what my second code does). So, how can I combine those two codes into one python code? I guess, it should be done somehow by importing twill, but how can it be done? Can a python code - the one that is being run by “Google App Engine” - import twill from somewhere on the internet? Or, perhaps, twill is already installed on “Google App Engine”?

    Read the article

  • How does a java compiler resolve a non-imported name

    - by gexicide
    Consider I use a type X in my java compilation unit from package foo.bar and X is not defined in the compilation unit itself nor is it directly imported. How does a java compiler resolve X now efficiently? There are a few possibilities where X could reside: X might be imported via a star import a.b.* X might reside in the same package as the compilation unit X might be a language type, i.e. reside in java.lang The problem I see is especially (2.). Since X might be a package-private type, it is not even required that X resides in a compilation unit that is named X.java. Thus, the compiler must look into all entries of the class path and search for any classes in a package foo.bar, it then must read every class that is in package foo.bar to check whether X is included. That sounds very expensive. Especially when I compile only a single file, the compiler has to read dozens of class files only to find a type X. If I use a lot of star imports, this procedure has to be repeated for a lot of types (although class files won't be read twice, of course). So is it advisable to import also types from the same package to speed up the compilation process? Or is there a faster method for resolving an unimported type X which I was not able to find?

    Read the article

< Previous Page | 34 35 36 37 38 39 40 41 42 43 44 45  | Next Page >