SWI-Prolog tokenize_atom/2 replacement?
- by Shark
What I need to do is to break atom to tokens. E. g.:
tokenize_string('Hello, World!', L).
would unify L=['Hello',',','World','!']. Exactly as tokenize_atom/2 do. But when I try to use tokenize_atom/2 with non-latin letters it fails. Is there any universal replacement or how I can write one? Thanks in advance.