I have an interesting
question about which programming idiom is easier to use for beginner developers writing concrete file parsing classes.
I'm developing an
open source library, which one of the main functionality is to parse plain text files and get structured information from them. All of the files contains the same kind of information, but can be in different formats like XML, plain text (each of them is structured differently), etc. There are a common set of information pieces which is the same in all (e.g. player names, table names, some id numbers)
There are formats which are very similar to each other, so it's possible to define a common Base class for them to facilitate concrete format parser implementations. So I can clearly define base classes like SplittablePlainTextFormat, XMLFormat, SeparateSummaryFormat, etc. Each of them hints the kind of structure they aim to parse. All of the concrete classes should have the same information pieces, no matter what.
To be useful at all, this library needs to define at least 30-40 of these parsers. A couple of them are more important than others (obviously the more popular formats).
Now my
question is, which is the best programming idiom to choose to facilitate the development of these concrete classes? Let me explain:
I think imperative programming is easy to follow even for beginners, because the flow is fixed, the statements just come one after another. Right now, I have this:
class SplittableBaseFormat:
def parse(self):
"Parses the body of the hand history, but first parse header if not yet parsed."
if not self.header_parsed:
self.parse_header()
self._parse_table()
self._parse_players()
self._parse_button()
self._parse_hero()
self._parse_preflop()
self._parse_street('flop')
self._parse_street('turn')
self._parse_street('river')
self._parse_showdown()
self._parse_pot()
self._parse_board()
self._parse_winners()
self._parse_extra()
self.parsed = True
So the concrete parser need to define these methods in order in any way they want. Easy to follow, but takes longer to implement each individual concrete parser.
So what about declarative? In this case Base classes (like SplittableFormat and XMLFormat) would do the heavy lifting based on regex and line/node number declarations in the concrete class, and concrete classes have no code at all, just line numbers and regexes, maybe other kind of rules.
Like this:
class SplittableFormat:
def parse_table():
"Parses TABLE_REGEX and get information"
# set attributes here
def parse_players():
"parses PLAYER_REGEX and get information"
# set attributes here
class SpecificFormat1(SplittableFormat):
TABLE_REGEX = re.compile('^(?P<table_name>.*) other info \d* etc')
TABLE_LINE = 1
PLAYER_REGEX = re.compile('^Player \d: (?P<player_name>.*) has (.*) in chips.')
PLAYER_LINE = 16
class SpecificFormat2(SplittableFormat):
TABLE_REGEX = re.compile(r'^Tournament #(\d*) (?P<table_name>.*) other info2 \d* etc')
TABLE_LINE = 2
PLAYER_REGEX = re.compile(r'^Seat \d: (?P<player_name>.*) has a stack of (\d*)')
PLAYER_LINE = 14
So if I want to make it possible for non-developers to write these classes the way to go seems to be the declarative way, however, I'm almost certain I can't eliminate the declarations of regexes, which clearly needs (senior :D) programmers, so should I care about this at all? Do you think it matters to choose one over another or doesn't matter at all? Maybe if somebody wants to work on this project, they will, if not, no matter which idiom I choose. Can I "convert" non-programmers to help developing these? What are your observations?
Other considerations:
Imperative will allow any kind of work; there is a simple flow, which they can follow but inside that, they can do whatever they want.
It would be harder to force a common interface with imperative because of this arbitrary implementations.
Declarative will be much more rigid, which is a bad thing, because formats might change over time without any notice.
Declarative will be harder for me to develop and takes longer time. Imperative is already ready to release.
I hope a nice discussion will happen in this thread about programming idioms regarding which to use when, which is better for
open source projects with different scenarios, which is better for wide range of developer skills.
TL; DR:
Parsing different file formats (plain text, XML)
They contains same kind of information
Target audience: non-developers, beginners
Regex probably cannot be avoided
30-40 concrete parser classes needed
Facilitate coding these concrete classes
Which idiom is better?
