Contributing to Transifex¶
Thanks for considering donating to the development of Transifex and our common goal of promoting, supporting, and advancing the Transifex framework.
Donating some of your time is the best support you could provide. Spreading a good word about Transifex is also very much appreciated. If you’re into coding, you can help us fix bugs and extend Transifex’s functionality. Great ways to contribute also include writing documentation and contributing translations of the software.
We’re passionate about helping Transifex users make the jump to contributing members of the community, so there are many ways you can help Transifex’s development:
- Report bugs and request features in our ticket tracker. Please read Reporting bugs, below, for the details on how we like our bug reports served up.
- Submit patches for new and/or fixed behavior. Please read Submitting code, below, for details on how to submit a patch.
- Join the transifex-devel mailing list and share your ideas for how to improve Transifex. We’re always open to suggestions, although we’re likely to be skeptical of large-scale suggestions without some code to back it up.
- Talk to us on IRC, on the Freenode network in the #transifex channel.
- Provide monetary donations to keep the lights on!
Reporting bugs¶
Well-written bug reports are incredibly helpful. However, there’s a certain amount of overhead involved in working with any bug tracking system, so your help in keeping our ticket tracker as useful as possible is appreciated. In particular:
- Do read the FAQ to see if your issue might be a well-known question.
- Do search the tracker to see if your issue has already been filed.
- Do ask on transifex-devel first if you’re not sure if what you’re seeing is a bug.
- Do write complete, reproducible, specific bug reports. Include as much information as you possibly can, complete with code snippets, test cases, etc. This means including a clear, concise description of the problem, and a clear set of instructions for replicating the problem. A minimal example that illustrates the bug in a nice small test case is the best possible bug report.
- Don’t use the ticket system to ask support questions. Use the transifex-devel list, or the #transifex IRC channel for that.
- Don’t use the ticket system to make large-scale feature requests. We like to discuss any big changes to Transifex’s core on the transifex-devel list before actually working on them.
- Don’t reopen issues that have been marked “wontfix”. This mark means that the decision has been made that we can’t or won’t fix this particular issue. If you’re not sure why, please ask on transifex-devel.
- Don’t use the ticket tracker for lengthy discussions, because they’re likely to get lost. If a particular ticket is controversial, please move discussion to transifex-devel.
- Don’t post to transifex-devel just to announce that you have filed a bug report. All the tickets are mailed to another list (transifex-updates), which is tracked by developers and triagers, so we see them as they are filed.
Submitting code¶
We’re always grateful for patches to Transifex’s code. Indeed, bug reports with associated patches will get fixed far more quickly than those without patches.
If this is the first time you are contributing code, you’ll need to sign the Transifex CLA.
Coding style¶
Here are some notes on the common style we use for stuff landing in Transifex.
Code follows the guidelines described in the standard Python Coding Style of PEP-8. Docstrings follow PEP-257.
Use the following style (indented, auto-concat) when writing long strings:
def lorem_ipsum(): description = _( "Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas " "nisl leo, suscipit sed, elementum ac, condimentum eget, augue. " "Pellentesque consequat posuere metus. Curabitur scelerisque faucibus " "massa. Vestibulum ut nisl ut leo cursus commodo." )
Model filtering should be in the form:
Module.query.filter_by(name='foo')
and not:
Module.query.filter(Module.name=='foo')
Committing¶
Commit messages should have a short first line and optionally more information after a blank newline:
Short (50 chars or less) summary of changes (#ticket)
More detailed explanatory text can follow after a blank line, if
necessary. Wrap to about 72 characters or so.
If there is a ticket associated, include it in the summary line. Prefix
line with "bugfix:" and "trivial:" for bugfixes and trivial commits.
- Bullet points are okay, too. Typically a hyphen or asterisk is used
for the bullet, preceded by a single space, with blank lines between
items.
- Use a hanging indent, like above.
Setup Hints for Common Editors¶
Configuring VIM for Transifex Code¶
To configure vim for editing Transifex code, make sure that it sets up the indentation level to 4 columns when editing Python code, that it expands ASCII TAB (code: 9) characters to spaces, and wraps text at 72 columns.
To enable these options only for *.py files, you can add the following options to your ~/.vimrc file:
if !exists("format_tx_python")
let format_tx_python = 1
" Formatting Python code for Transifex.
autocmd BufNewFile,BufRead *.py set autoindent showmatch
autocmd BufNewFile,BufRead *.py set formatoptions=tcq2l
autocmd BufNewFile,BufRead *.py set textwidth=72 shiftwidth=4
autocmd BufNewFile,BufRead *.py set softtabstop=4 tabstop=8 expandtab
endif
Configuring Emacs for Transifex Code¶
To configure Emacs for editing Transifex code, you can set up a hook function that runs whenever python-mode is enabled. Setting up the column for wrapping text, and the use of only ASCII SPACE characters for indentation is easy. Just add the following Emacs Lisp snippet in your Emacs startup file:
;; Formatting Python code for Transifex.
(defun local/python-mode-options ()
"My local configuration tweaks for `python-mode'."
(setq fill-column 72 ;text wrap column
python-indent 4 ;use 4-column indentation for Python
indent-tabs-mode nil ;use only SPC for indentation
next-line-add-newlines nil)) ;don't add newlines at end-of-file
;; When python-mode loads, call our hook function.
(eval-after-load "python"
'(add-hook 'python-mode 'local/python-mode-options))
Changing the model¶
As with all software involving a database, changes to Transifex’s models should be handled with care. Production instances of Transifex rely on the stability of the model when upgrading, and a safe upgrade path should be given to everyone currently running a Transifex instance.
For this reason we usually discuss model changes on the mailing list, especially when they’re big ones.
New model fields should follow the style of the current fields and include any help texts attributes to them, if needed.
A commit which changes the models needs to include fixtures that work with the new model. We already have some sample data for folks to try out in txcommon/fixtures/sample_data.json.
Typical model change scenario¶
When model modifications are introduced, the following steps should be followed to have the fixture updated:
Flush, create tables and load data:
./manage.py flush ./manage.py syncdb ./manage.py loaddata txcommon/fixtures/sample_data.json
Make changes to the models.
Probably a migration will be required.
Starting with Release 0.7, Transifex uses South for managing the schema migrations of models. To get the migration script related to your model change run:
./manage.py startmigration <app_name> <migration_name> --auto
Note that a new python script with the <migration_name> will be created under the <app_name>/migrations/ dir. As it’s an automatic detection, please, have a look at it to verify that it is accurately reflects your changes. After reviewing it, you can test it running:
./manage.py migrate <app_name> --db-dry-run
In complex changes the South application might not be able to proceed automatically (for example when adding a non-null column). In this case you can delete the column and re-add it, add it as null and change it, etc. South also supports to manual migrations and more information can be found at http://south.aeracode.org/wiki/Documentation.
Once your migration passes through the above command without errors, you can proceed with the migration in the database by omitting the option –db-dry-run:
./manage.py migrate <app_name>
With the new DB schema in place, dump the data again:
./manage.py dumpdata --indent=2 projects releases codebases \ tarball vcs sites > txcommon/fixtures/sample_data.json
Try the new DB fixture:
./manage.py flush --noinput ./manage.py syncdb --noinput ./manage.py loaddata txcommon/fixtures/sample_data.json ./manage.py loaddata txcommon/fixtures/sample_users.json
Open up your browser to check that everything looks OK and that your changes have been included as you expected.
Update the documentation if needed. Commit all changes together.
Updating translation files (PO files)¶
To extract new messages from the source code and update the PO files of Transifex it is necessary to run the following command and then commit the changes:
./manage.py txmakemessages --all
How to add support for a new file format¶
This is a small tutorial to show you how to write a handler for a new file format in Transifex. This covers most of the basics to get you started with writing new file handlers. However, for more details, you can always go through the source code of the file handlers already available in Transifex. They can be found in the directory transifex/resources/formats of Transifex’s source code.
Sample file¶
We’ll write a sample handler to parse and compile the file shown below.
KEY1=Hello, world
KEY2=This contains special chars \n \t " '
Sample Handler code¶
You can use the sample code below as a base template to start writing a handler for a new file format. We’ll write a new file transifex/resources/formats/xyz.py with the content below. The code is accompanied with detailed comments to help you understand better.
# -*- coding: utf-8 -*-
import re, codecs
from transifex.resources.models import SourceEntity
from transifex.resources.formats.utils.decorators import *
from transifex.resources.formats.utils.hash_tag import hash_tag
from transifex.resources.formats.core import Handler, ParseError, CompileError
from transifex.resources.formats.compilation import Compiler,\
SimpleCompilerFactory
class SampleParseError(ParseError):
pass
class SampleCompileError(CompileError):
pass
class SampleCompiler(Compiler):
def _compile(self, content):
"""Internal compile function.
Subclasses must override this method, if they need to change
the compile behavior.
Args:
content: The content (template) of the resource.
"""
"""In our case, we won't need to customize this method.
The default implementation does the job of replacing
md5 patterns in the template with the corresponding
translations."""
super(SampleCompiler, self)._compile(content)
def _post_compile(self, content=None):
"""Do any work after the compilation process."""
super(SampleCompiler, self)._post_compile(content)
class SampleHandler(SimpleCompilerFactory, Handler):
"""Some info and properties for the handler."""
name = 'Sample handler'
format = "Sample handler .xyz (*.xyz)"
#method_name uniquely identifies this handler among other handlers
method_name = 'XYZ'
format_encoding = 'UTF-8'
"""Bind the error handlers"""
HandlerCompilerError = SampleCompileError
HandlerParseError = SampleParseError
"""Bind the compiler class to the Handler"""
CompilerClass = SampleCompiler
def _parse(self, is_source, lang_rules):
"""The actual functions that parses the content.
Formats need to override this to provide the desired behavior.
Two stringsets are available to subclasses:
- self.stringset to save the translated strings
- self.suggestions to save suggested translations
Args:
is_source: Flag to determine if this is a source file or not.
lang_rules: rules for the language
Returns:
An object which, when used as an argument in
`self._create_template()`, the template for the resource
is generated.
"""
template = u""
context = ""
for line in self.content.splitlines():
#split line to get key, value or source, translation pair
#In this case, the above file format is key-value based
key, value = line.split('=', 1)
"""
During importing source files, we have to generate a template
in which values/translations are replaced with an md5 pattern.
This template is used during compiling the file for download.
We generate a unique md5 pattern for (key/source string, context)
pair.
"""
if is_source:
"""
During importing source file,
generate an md5 pattern and replace the value only if
value has some useful data (and not just whitespaces).
Else, save the line as it is in the template.
"""
if value.strip():
template += key + '=' + '%s_tr' % hash_tag(key, '')
else:
template += line
elif not value.strip():
"""Similarly, during importing a translation file, if
value doesn't contain any useful data like above,
then move to the next line."""
continue
"""This adds the (key, value, context) to self.stringset
to be saved to the database."""
self._add_translation_string(key, value, context=context)
return template
def _escape(self, s):
"""Escaping during compilation"""
"""Let's say we'll escape \n and \t"""
return (s.replace('\n', r'\n')
.replace('\t', r'\t')
)
def _unescape(self, s):
"""Unescaping durng importing"""
"""Let's say we'll unescape \\n and \\t"""
return (s.replace(r'\t', '\t')
.replace(r'\n', '\n')
)
Integrate SampleHandler with Transifex¶
In transifex/settings/70-translation.conf, add the following to I18N_METHODS dictionary:
I18N_METHODS = { ... 'XYZ': { """This will appear in the i18n types selection menu""" 'description': 'XYZ file', 'mimetype': 'text/plain', 'file-extensions': '.xyz' }, }
Again in transifex/settings/70-translation.conf, we have to add the following to I18N_HANDLER_CLASS_NAMES dictionary:
I18N_HANDLER_CLASS_NAMES = { ... 'XYZ': 'transifex.resources.formats.xyz.SampleHandler', }
And we are done! Now, we restart the server (Django test server or whatever). Now, we’ll be able to see “XYZ file” in the list of supported i18n types during resource creation. When we create our resource with the above sample file and “XYZ file” chosen as i18n type, the following template will be stored in the database for use during compilation.
KEY1=50311154357828e0ad520eace59ef0e5_tr KEY2=122ea8bfd5e1babcc99eca728985bc6b_tr
Let’s say, we have translated a string “Hello, world” in Portuguese as “Olá mundo,”. When the Portuguese translation file is downloaded, the file will look like:
KEY1=Olá mundo, KEY2=
During the compilation process, the md5 patterns were replaced by their corresponding translations, ‘Olá mundo,’ for KEY1 and ‘’ for KEY2.
Various Development Tips¶
Useful URLs¶
Transifex:
- http://help.transifex.com/technical/contributing.html
- http://help.transifex.com/
- http://code.transifex.com/
- http://code.transifex.com/transifex
- http://code.transifex.com/transifex-client
Python & Django:
~/.bashrc¶
# User specific aliases and functions
bind '"\e[A"':history-search-backward # first letters of command + up/down
bind '"\e[B"':history-search-forward
alias mg="python manage.py"
alias ll='ls -l --color=tty'
export WORKON_HOME=~mits/devel/env/
source /usr/bin/virtualenvwrapper.sh
HISTSIZE=20000
HISTFILESIZE=999999
shopt -s histappend
Useful packages¶
- bash-completion
- xchat-gnome
- ipython, python-nose
- skype, pidgin
- git
- meld
- ipython
Minor ones:
- eclipse + pydev
- regexxer # mass search & replace, recursive in dirs
- shutter # screenshots
- freemind # mind mapping for notes, brainstorming
- istanbul # screencasts
- mgopen fonts # Tx logo etc
Testing¶
You first have to have all the necessary packages installed. Details can be found at Install manually.
Then, give the command:
cd transifex/transifex
to enter the directory with the transifex source code.
You can test any app and addon with:
./manage.py test <app_name>
If you want to execute a specific test case for the app, you can use:
./manage.py test <app_name>.<TestCase>
For example, to test the model for the resources app (the test case is in transifex/resources/tests/models.py), you should use the command:
./manage.py test resources.ResourcesModelTests
For details, see https://docs.djangoproject.com/en/1.2/topics/testing/.
Other useful tips¶
Add exc_info=True when logging from inside an exception handler
Find string: find . -type f -iname '*py' | xargs grep <str>
Add the following lines to settings.py to create the test database in memory only:
if 'test' in sys.argv: DATABASES = { 'default': { 'NAME': os.path.join(PROJECT_PATH, 'transifex.db.sqlite'), 'ENGINE': 'django.db.backends.sqlite3', }, }
Add the following line to settings.py to skip running migrations for tests:
SOUTH_TESTS_MIGRATE = False
Insert the line:
import ipdb; ipdb.set_trace();
to interrupt the execution of the program at the specified position. See http://docs.python.org/library/pdb.html for what you can do then.
Cross Site Request Forgery protection¶
Transifex is protected against CSRF attacks by ensuring that GET requests are side-effect free. This means that every action inside Transifex that has an effect on the database, session, etc. are not being done over a GET request. With Django’s CSRF Middleware enabled, attackers are prevented from meddling with a Transifex user’s settings.
- Examples of CSRF-vulnerable elements include::
- a link which calls /logout over GET to log the user out
- a ‘pt_BR’ link to change the page’s language to Brazilian Portuguese and save this setting to the user’s cookie
- a simple AJAX request to lock a file.
- Examples of safe actions include::
- searching through the website for a string
- pagination
If you are adding some code in Transifex which has a side-effect, always require POST.
Frequently Asked Questions¶
I submitted a bug fix several weeks ago. Why are you ignoring my patch?¶
Don’t worry: We’re not ignoring you!
It’s important to understand there is a difference between “a ticket is being ignored” and “a ticket has not been attended to yet.” Our ticket system contains hundreds of open tickets, with various degrees of impact on end-user functionality, and Transifex’s developers have to review and prioritize.
On top of that: the people who work on Transifex are all volunteers. As a result, the amount of time that we have to work on the framework is limited and will vary from week to week depending on our spare time. If we’re busy, we may not be able to spend as much time on Transifex as we might want.
Besides, if your feature request stands no chance of inclusion in Transifex, we won’t ignore it – we’ll just close the ticket. So if your ticket is still open, it doesn’t mean we’re ignoring you; it just means we haven’t had time to look at it yet.
Next Section: The big, scary Transifex One-Dot-Zero »