Search Results

Search found 178 results on 8 pages for 'sqlalchemy'.

Page 4/8 | < Previous Page | 1 2 3 4 5 6 7 8  | Next Page >

  • SQLAlchemy Custom Type Which Contains Multiple Columns

    - by Kekoa
    I would like to represent a datatype as a single column in my model, but really the data will be stored in multiple columns in the database. I cannot find any good resources on how to do this in SQLAlchemy. I would like my model to look like this(this is a simplified example using geometry instead of my real problem which is harder to explain): class 3DLine(DeclarativeBase): start_point = Column(my.custom.3DPoint) end_point = Column(my.custom.3DPoint) This way I could assign an object with the (x, y, z) components of the point at once without setting them individually. If I had to separate each component, this could get ugly, especially if each class has several of these composite objects. I would combine the values into one encoded field except that I need to query each value separately at times. I was able to find out how to make custom types using a single column in the documentation. But there's no indication that I can map a single type to multiple columns. I suppose I could accomplish this by using a separate table, and each column would be a foreign key, but in my case I don't think it makes sense to have a one to one mapping for each point to a separate table, and this still does not give the ability to set the related values all at once.

    Read the article

  • Faster Insertion of Records into a Table with SQLAlchemy

    - by Kyle Brandt
    I am parsing a log and inserting it into either MySQL or SQLite using SQLAlchemy and Python. Right now I open a connection to the DB, and as I loop over each line, I insert it after it is parsed (This is just one big table right now, not very experienced with SQL). I then close the connection when the loop is done. The summarized code is: log_table = schema.Table('log_table', metadata, schema.Column('id', types.Integer, primary_key=True), schema.Column('time', types.DateTime), schema.Column('ip', types.String(length=15)) .... engine = create_engine(...) metadata.bind = engine connection = engine.connect() .... for line in file_to_parse: m = line_regex.match(line) if m: fields = m.groupdict() pythonified = pythoninfy_log(fields) #Turn them into ints, datatimes, etc if use_sql: ins = log_table.insert(values=pythonified) connection.execute(ins) parsed += 1 My two questions are: Is there a way to speed up the inserts within this basic framework? Maybe have a Queue of inserts and some insertion threads, some sort of bulk inserts, etc? When I used MySQL, for about ~1.2 million records the insert time was 15 minutes. With SQLite, the insert time was a little over an hour. Does that time difference between the db engines seem about right, or does it mean I am doing something very wrong?

    Read the article

  • Need help with joins in sqlalchemy

    - by Steve
    I'm new to Python, as well as SQL Alchemy, but not the underlying development and database concepts. I know what I want to do and how I'd do it manually, but I'm trying to learn how an ORM works. I have two tables, Images and Keywords. The Images table contains an id column that is its primary key, as well as some other metadata. The Keywords table contains only an id column (foreign key to Images) and a keyword column. I'm trying to properly declare this relationship using the declarative syntax, which I think I've done correctly. Base = declarative_base() class Keyword(Base): __tablename__ = 'Keywords' __table_args__ = {'mysql_engine' : 'InnoDB'} id = Column(Integer, ForeignKey('Images.id', ondelete='CASCADE'), primary_key=True) keyword = Column(String(32), primary_key=True) class Image(Base): __tablename__ = 'Images' __table_args__ = {'mysql_engine' : 'InnoDB'} id = Column(Integer, primary_key=True, autoincrement=True) name = Column(String(256), nullable=False) keywords = relationship(Keyword, backref='image') This represents a many-to-many relationship. One image can have many keywords, and one keyword can relate back to many images. I want to do a keyword search of my images. I've tried the following with no luck. Conceptually this would've been nice, but I understand why it doesn't work. image = session.query(Image).filter(Image.keywords.contains('boy')) I keep getting errors about no foreign key relationship, which seems clearly defined to me. I saw something about making sure I get the right 'join', and I'm using 'from sqlalchemy.orm import join', but still no luck. image = session.query(Image).select_from(join(Image, Keyword)).\ filter(Keyword.keyword == 'boy') I added the specific join clause to the query to help it along, though as I understand it, I shouldn't have to do this. image = session.query(Image).select_from(join(Image, Keyword, Image.id==Keyword.id)).filter(Keyword.keyword == 'boy') So finally I switched tactics and tried querying the keywords and then using the backreference. However, when I try to use the '.images' iterating over the result, I get an error that the 'image' property doesn't exist, even though I did declare it as a backref. result = session.query(Keyword).filter(Keyword.keyword == 'boy').all() I want to be able to query a unique set of image matches on a set of keywords. I just can't guess my way to the syntax, and I've spent days reading the SQL Alchemy documentation trying to piece this out myself. I would very much appreciate anyone who can point out what I'm missing.

    Read the article

  • SQLAlchemy custom query column

    - by thrillerator
    I have a declarative table defined like this: class Transaction(Base): __tablename__ = "transactions" id = Column(Integer, primary_key=True) account_id = Column(Integer) transfer_account_id = Column(Integer) amount = Column(Numeric(12, 2)) ... The query should be: SELECT id, (CASE WHEN transfer_account_id=1 THEN -amount ELSE amount) AS amount FROM transactions WHERE account_id = 1 OR transfer_account_id = 1 My code is: query = Transaction.query.filter_by(account_id=1, transfer_account_id=1) query = query.add_column(func.case(...).label("amount") But it doesn't replace the amount column. Been trying to do this with for hours and I don't want to use raw SQL.

    Read the article

  • SQLAlchemy: select over multiple tables

    - by ahojnnes
    Hi, I wanted to optimize my database query: link_list = select( columns=[link_table.c.rating, link_table.c.url, link_table.c.donations_in], whereclause=and_( not_(link_table.c.id.in_( select( columns=[request_table.c.recipient], whereclause=request_table.c.donator==donator.id ).as_scalar() )), link_table.c.id!=donator.id, ), limit=20, ).execute().fetchall() and tried to merge those two selects in one query: link_list = select( columns=[link_table.c.rating, link_table.c.url, link_table.c.donations_in], whereclause=and_( link_table.c.active==True, link_table.c.id!=donator.id, request_table.c.donator==donator.id, link_table.c.id!=request_table.c.recipient, ), limit=20, order_by=[link_table.c.rating.desc()] ).execute().fetchall() the database-schema looks like: link_table = Table('links', metadata, Column('id', Integer, primary_key=True, autoincrement=True), Column('url', Unicode(250), index=True, unique=True), Column('registration_date', DateTime), Column('donations_in', Integer), Column('active', Boolean), ) request_table = Table('requests', metadata, Column('id', Integer, primary_key=True, autoincrement=True), Column('recipient', Integer, ForeignKey('links.id')), Column('donator', Integer, ForeignKey('links.id')), Column('date', DateTime), ) There are several links (donator) in request_table pointing to one link in the link_table. I want to have links from link_table, which are not yet "requested". But this does not work. Is it actually possible, what I'm trying to do? If so, how would you do that? Thank you very much in advance!

    Read the article

  • SQLAlchemy - relationship limited on more than just the foreign key

    - by Marian
    I have a wiki db layout with Page and Revisions. Each Revision has a page_id referencing the Page, a page relationship to the referenced page; each Page has a all_revisions relationship to all its revisions. So far so common. But I want to implement different epochs for the pages: If a page was deleted and is recreated, the new revisions have a new epoch. To help find the correct revisions, each page has a current_epoch field. Now I want to provide a revisions relation on the page that only contains its revisions, but only those where the epochs match. This is what I've tried: revisions = relationship('Revision', primaryjoin = and_( 'Page.id == Revision.page_id', 'Page.current_epoch == Revision.epoch', ), foreign_keys=['Page.id', 'Page.current_epoch'] ) Full code (you may run that as it is) However this always raises ArgumentError: Could not determine relationship direction for primaryjoin condition ...`, I've tried all I had come to mind, it didn't work. What am I doing wrong? Is this a bad approach for doing this, how could it be done other than with a relationship?

    Read the article

  • Setting 0/1-to-1 relationship in SQLAlchemy?

    - by Timmy
    is there a proper way of setting up a 0/1-to-1 relationship? i want to be able to check if the related item exists without creating it: if item.relationship is None: item.relationship = item2() but it creates the insert statements on the if statement

    Read the article

  • Hooking entity creation in SQLAlchemy

    - by Dan Ellis
    I want to write a SessionExtension that fires a 'Foo-created' event or 'Bar-created' event every time a new Foo or new Bar is committed to the database. However, once inside the after_commit method, I don't know where to find which entities have been committed. Where do I get this information?

    Read the article

  • How to create a non-persistent Elixir/SQLAlchemy object?

    - by siebert
    Hi, because of legacy data which is not available in the database but some external files, I want to create a SQLAlchemy object which contains data read from the external files, but isn't written to the database if I execute session.flush() My code looks like this: try: return session.query(Phone).populate_existing().filter(Phone.mac == ident).one() except: return self.createMockPhoneFromLicenseFile(ident) def createMockPhoneFromLicenseFile(self, ident): # Some code to read necessary data from file deleted.... phone = Phone() phone.mac = foo phone.data = bar phone.state = "Read from legacy file" phone.purchaseOrderPosition = self.getLegacyOrder(ident) # SQLAlchemy magic doesn't seem to work here, probably because we don't insert the created # phone object into the database. So we set the id fields manually. phone.order_id = phone.purchaseOrderPosition.order_id phone.order_position_id = phone.purchaseOrderPosition.order_position_id return phone Everything works fine except that on a session.flush() executed later in the application SQLAlchemy tries to write the created Phone object to the database (which fortunatly doesn't succeed, because phone.state is longer than the data type allows), which breaks the function which issues the flush. Is there any way to prevent SQLAlchemy from trying to write such an object? Ciao, Steffen

    Read the article

  • SQLAlchemy sessions - DetachedInstanceError?

    - by benjaminhkaiser
    I have a function that attempts to take a list of usernames, look each one up in a user table, and then add them to a membership table. If even one username is invalid, I want the entire list to be rolled back, including any users that have already been processed. I thought that using sessions was the best way to do this but I'm running into a DetachedInstanceError: DetachedInstanceError: Instance <Organization at 0x7fc35cb5df90> is not bound to a Session; attribute refresh operation cannot proceed Full stack trace is here. The error seems to trigger when I attempt to access the user (model) object that is returned by the query. From my reading I understand that it has something to do with there being multiple sessions, but none of the suggestions I saw on other threads worked for me. Code is below: def add_members_in_bulk(organization_eid, users): """Add users to an organization in bulk - helper function for add_member()""" """Returns "success" on success and id of first failed student on failure""" session = query_session.get_session() session.begin_nested() users = users.split('\n') for u in users: try: user = user_lookup.by_student_id(u) except ObjectNotFoundError: session.rollback() return u if user: membership.add_user_to_organization( user.entity_id, organization_eid, '', [] ) session.flush() session.commit() return 'success' here's the membership.add_user_to_organization: def add_user_to_organization(user_eid, organization_eid, title, tag_ids): """Add a User to an Organization with the given title""" user = user_lookup.by_eid(user_eid) organization = organization_lookup.by_eid(organization_eid) new_membership = OrganizationMembership( organization_eid=organization.entity_id, user_eid=user.entity_id, title=title) new_membership.tags = [get_tag_by_id(tag_id) for tag_id in tag_ids] crud.add(new_membership) and here is the lookup by ID query: def by_student_id(student_id, include_disabled=False): """Get User by RIN""" try: return get_query_set(include_disabled).filter(User.student_id == student_id).one() except NoResultFound: raise ObjectNotFoundError("User with RIN %s does not exist." % student_id)

    Read the article

  • Default subclass objects in Sqlalchemy?

    - by Timmy
    im using the example from the pylons book orm.mapper(Comment, comment_table) orm.mapper(Tag, tag_table) orm.mapper(Nav, nav_table, polymorphic_on=nav_table.c.type, polymorphic_identity='nav') orm.mapper(Section, section_table, inherits=Nav, polymorphic_identity='section') orm.mapper(Page, page_table, inherits=Nav, polymorphic_identity='page', properties={ 'comments':orm.relation(Comment, backref='page', cascade='all'), 'tags':orm.relation(Tag, secondary=pagetag_table) }) i am mostly copying from this, but is there a simple way have a default Page that gets referenced, but if users requests a change, create a new Page object? thanks i want something similar to this class DefaultPage(Page): __init__(self): self.a = a self.b = b self.c = c orm.mapper(DefaultPage, None, inherits=Nav, yada yada )

    Read the article

  • Sql Alchemy Duplicated Commit

    - by PythonWolf
    Good Morning i'm currently facing a problem in my Cherrypy application. Im my own custom session module , anyway when performing session.add() The exact same object gets updated Twice. cherrypy.request.SessionManager.user_data = user try: db_session.add(cherrypy.request.SessionManager) db_session.commit() Will Return 2011-06-21 09:16:48,991 INFO sqlalchemy.engine.base.Engine.0x...04cL BEGIN (implicit) 2011-06-21 09:16:49,015 INFO sqlalchemy.engine.base.Engine.0x...04cL SELECT ..... FROM "Clients_Users" WHERE "Clients_Users".username = %(username_1)s AND "Clients_Users".password = %(password_1)s LIMIT 1 OFFSET 0 2011-06-21 09:16:49,015 INFO sqlalchemy.engine.base.Engine.0x...04cL {'password_1': '123', 'username_1': u'1'} 2011-06-21 09:16:49,047 INFO sqlalchemy.engine.base.Engine.0x...04cL UPDATE "SYS_Sessions" SET user_data=%(user_data)s WHERE "SYS_Sessions".id = %(SYS_Sessions_id)s 2011-06-21 09:16:49,067 INFO sqlalchemy.engine.base.Engine.0x...04cL {'SYS_Sessions_id': 92L, 'user_data': } 2011-06-21 09:16:49,071 INFO sqlalchemy.engine.base.Engine.0x...04cL COMMIT 2011-06-21 09:16:49,093 INFO sqlalchemy.engine.base.Engine.0x...04cL BEGIN (implicit) 2011-06-21 09:16:49,095 INFO sqlalchemy.engine.base.Engine.0x...04cL UPDATE "SYS_Sessions" SET user_data=%(user_data)s WHERE "SYS_Sessions".id = %(SYS_Sessions_id)s 2011-06-21 09:16:49,095 INFO sqlalchemy.engine.base.Engine.0x...04cL {'SYS_Sessions_id': 92L, 'user_data': } 2011-06-21 09:16:49,108 INFO sqlalchemy.engine.base.Engine.0x...04cL COMMIT As Anyone seen this before ? P.S This doesn't happen in the rest of the modules i have made.

    Read the article

  • SQLAlchemy: an efficient/better select by primary keys?

    - by hadrien
    Yet another newbie question.. Let's say I have an user table in declarative mode: class User(Base): __tablename__ = 'user' id = Column(u'id', Integer(), primary_key=True) name = Column(u'name', String(50)) When I have a list of users identifiers, I fetch them from db with: user_ids = [1, 2, 3, 4, 5] users = Session.query(User).filter(User.id.in_(user_ids)).all() I dislike using in_ because I think I learned it has bad performance on indexed fields (is that true/false?). Anyway, is there a better way doing that query? Thanks!

    Read the article

  • Return more then One field from database SQLAlchemy

    - by David Neudorfer
    This line: used_emails = [row.email for row in db.execute(select([halo4.c.email], halo4.c.email!=''))] Returns: ['[email protected]', '[email protected]', '[email protected]', '[email protected]', '[email protected]'] I use this to find a match: if recipient in used_emails: If it finds a match I need to pull another field (halo4.c.code) from the database in the same row. Any suggestions on how to do this?

    Read the article

  • Using SQLAlchemy, how can I return a count with multiple columns

    - by Andy
    I am attempting to run a query like this: SELECT comment_type_id, name, count(comment_type_id) FROM comments, commenttypes WHERE comment_type_id=commenttypes.id GROUP BY comment_type_id Without the join between comments and commenttypes for the name column, I can do this using: session.query(Comment.comment_type_id,func.count(Comment.comment_type_id)).group_by(Comment.comment_type_id).all() However, if I try to do something like this, I get incorrect results: session.query(Comment.comment_type_id, Comment.comment_type, func.count(Comment.comment_type_id)).group_by(Comment.comment_type_id).all() I have two problems with the results: (1, False, 82920) (2, False, 588) (3, False, 4278) (4, False, 104370) Problems: The False is not correct The counts are wrong My expected results are: (1, 'Comment Type 1', 13820) (2, 'Comment Type 2', 98) (3, 'Comment Type 2', 713) (4, 'Comment Type 2', 17395) How can I adjust my command to pull the correct name value and the correct count?

    Read the article

  • sqlalchemy natural sorting

    - by teggy
    Currently, i am querying with this code: meta.Session.query(Label).order_by(Label.name).all() and it returns me objects sorted by Label.name in this manner ['1','7','1a','5c']. Is there a way i can have the objects returned in the order with their Label.name sorted like this ['1','1a','5c','7'] Thanks!

    Read the article

  • How to map one class against multiple tables with SQLAlchemy?

    - by tote
    Lets say that I have a database structure with three tables that look like this: items - item_id - item_handle attributes - attribute_id - attribute_name item_attributes - item_attribute_id - item_id - attribute_id - attribute_value I would like to be able to do this in SQLAlchemy: item = Item('item1') item.foo = 'bar' session.add(item) session.commit() item1 = session.query(Item).filter_by(handle='item1').one() print item1.foo # => 'bar' I'm new to SQLAlchemy and I found this in the documentation (http://www.sqlalchemy.org/docs/05/mappers.html#mapping-a-class-against-multiple-tables): j = join(items, item_attributes, items.c.item_id == item_attributes.c.item_id). \ join(attributes, item_attributes.c.attribute_id == attributes.c.attribute_id) mapper(Item, j, properties={ 'item_id': [items.c.item_id, item_attributes.c.item_id], 'attribute_id': [item_attributes.c.attribute_id, attributes.c.attribute_id], }) It only adds item_id and attribute_id to Item and its not possible to add attributes to Item object. Is what I'm trying to achieve possible with SQLAlchemy? Is there a better way to structure the database to get the same behaviour of "dynamic columns"?

    Read the article

  • Filtering SQLAlchemy query on attribute_mapped_collection field of relationship

    - by bsa
    I have two classes, Tag and Hardware, defined with a simple parent-child relationship (see the full definition at the end). Now I want to filter a query on Tag using the version field in Hardware through an attribute_mapped_collection, eg: def get_tags(order_code=None, hardware_filters=None): session = Session() query = session.query(Tag) if order_code: query = query.filter(Tag.order_code == order_code) if hardware_filters: for k, v in hardware_filters.iteritems(): query = query.filter(getattr(Tag.hardware, k).version == v) return query.all() But I get: AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Tag.hardware has an attribute 'baseband The same thing happens if I strip it back by hard-coding the attribute, eg: query.filter(Tag.hardware.baseband.version == v) I can do it this way: query = query.filter(Tag.hardware.any(artefact=k, version=v)) But why can't I filter directly through the attribute? Class definitions class Tag(Base): __tablename__ = 'tag' tag_id = Column(Integer, primary_key=True) order_code = Column(String, nullable=False) version = Column(String, nullable=False) status = Column(String, nullable=False) comments = Column(String) hardware = relationship( "Hardware", backref="tag", collection_class=attribute_mapped_collection('artefact'), ) __table_args__ = ( UniqueConstraint('order_code', 'version'), ) class Hardware(Base): __tablename__ = 'hardware' hardware_id = Column(Integer, primary_key=True) tag_id = Column(String, ForeignKey('tag.tag_id')) product_id = Column(String, nullable=True) artefact = Column(String, nullable=False) version = Column(String, nullable=False)

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8  | Next Page >